Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move rust-sdk wasm artifact into bundles directory #28624

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 12 additions & 8 deletions scripts/deploy.py
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note this script isn't used for most deployments, its pretty much only used for app.element.io, staging.element.io, and even there it will not be used anymore soon once we switch to CFP.

Original file line number Diff line number Diff line change
Expand Up @@ -53,11 +53,15 @@ def move_bundles(source, dest):
renames = {}
for f in os.listdir(source):
dst = os.path.join(dest, f)
# If this bundle already exists, then likely the same bundle is used in multiple
# versions of the app. There's no need to copy the bundle again; however we do
# update the mtime so that the cleanup script knows that the bundle is still used.
if os.path.exists(dst):
print(
"Skipping bundle. The bundle includes '%s' which we have previously deployed."
"Skipping bundle '%s' which we have previously deployed."
% f
)
os.utime(dst)
else:
renames[os.path.join(source, f)] = dst

Expand Down Expand Up @@ -101,20 +105,20 @@ def fetch(self, tarball, extract_path):
def is_within_directory(directory, target):
abs_directory = os.path.abspath(directory)
abs_target = os.path.abspath(target)

prefix = os.path.commonprefix([abs_directory, abs_target])

return prefix == abs_directory

def safe_extract(tar, path=".", members=None, *, numeric_owner=False):
for member in tar.getmembers():
member_path = os.path.join(path, member.name)
if not is_within_directory(path, member_path):
raise Exception("Attempted Path Traversal in Tar File")
tar.extractall(path, members, numeric_owner=numeric_owner)

tar.extractall(path, members, numeric_owner=numeric_owner)


safe_extract(tar, extract_path)
finally:
if self.should_clean and downloaded:
Expand Down
30 changes: 22 additions & 8 deletions webpack.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -573,7 +573,7 @@ module.exports = (env, argv) => {
// This exports our CSS using the splitChunks and loaders above.
new MiniCssExtractPlugin({
filename: "bundles/[fullhash]/[name].css",
chunkFilename: "bundles/[fullhash]/[name].css",
chunkFilename: "bundles/[fullhash]/[name].chunk.css",
ignoreOrder: false, // Enable to remove warnings about conflicting order
}),

Expand Down Expand Up @@ -688,16 +688,30 @@ module.exports = (env, argv) => {
output: {
path: path.join(__dirname, "webapp"),

// The generated JS (and CSS, from the extraction plugin) are put in a
// unique subdirectory for the build. There will only be one such
// 'bundle' directory in the generated tarball; however, hosting
// servers can collect 'bundles' from multiple versions into one
// directory and symlink it into place - this allows users who loaded
// an older version of the application to continue to access webpack
// chunks even after the app is redeployed.
// There are a lot of assets that need to be kept in sync with each other
// (once a user loads one version of the app, they need to keep being served
// assets for that version).
//
// To deal with this, we try to put as many as possible of the referenced assets
// into a build-specific subdirectory. This includes generated javascript, as well
// as CSS extracted by the MiniCssExtractPlugin (see config above).
//
// Hosting servers can then collect 'bundles' from multiple versions
// into one directory, and continue to serve them even after a new version is deployed.
// This allows users who loaded an older version of the application to continue to
// access assets even after the app is redeployed.
//
// See `scripts/deploy.py` for a script which manages the deployment in this way.
filename: "bundles/[fullhash]/[name].js",
chunkFilename: "bundles/[fullhash]/[name].js",
webassemblyModuleFilename: "bundles/[fullhash]/[modulehash].wasm",

// Asset modules include things like the WASM for matrix-rust-sdk-crypto. Again
// these need keeping in sync with the rest of the application. Unfortunately
// `fullhash` (ie, a build-specific hash) is not available for these, so the best
// we can do is put them in their own directory. On the other hand, that does mean
// that such assets can be cached across Element versions.
assetModuleFilename: "bundles/[contenthash]/[name][ext]",
t3chguy marked this conversation as resolved.
Show resolved Hide resolved
},

// configuration for the webpack-dev-server
Expand Down
Loading