{"id":2823,"date":"2022-02-27T17:17:31","date_gmt":"2022-02-27T17:17:31","guid":{"rendered":"https:\/\/dft.wiki\/?p=2823"},"modified":"2026-06-08T21:54:59","modified_gmt":"2026-06-09T01:54:59","slug":"aws-s3-glacier-cheat-sheet-the-ultimate-backup-layer","status":"publish","type":"post","link":"https:\/\/dft.wiki\/?p=2823","title":{"rendered":"AWS S3 Glacier Cheat Sheet (The Ultimate Backup Layer)"},"content":{"rendered":"<p>S3 Glacier is an AWS storage service intended strictly as a last-resort backup, only if everything else fails.<\/p>\n<p>Files stored in this service are rarely accessed and retrieval comes with a cost. In other words, do not expect to get them back easily.<\/p>\n<p>If you do need to retrieve your files, be prepared to wait hours or even days before they are available for download.<\/p>\n<p>One important tip: avoid sending small files. Always zip them into large archives, but make sure to split them into pieces smaller than 4 GB. Doing this upfront will save you from a painful multipart upload process (not recommended).<\/p>\n<p>See the official documentation for CLI commands at [<a href=\"https:\/\/docs.aws.amazon.com\/cli\/latest\/reference\/glacier\/index.html\">Link<\/a>].<\/p>\n<hr \/>\n<p><strong>CREATING A VAULT<\/strong><\/p>\n<p>Vaults in S3 Glacier are the equivalent of Buckets in S3.<\/p>\n<pre>aws glacier create-vault --vault-name <strong>my-glacier<\/strong> --account-id -<\/pre>\n<p><strong>UPLOADING AN ARCHIVE SMALLER THAN 4 GB<\/strong><\/p>\n<pre>aws glacier upload-archive --account-id - --vault-name <strong>my-glacier<\/strong> --body <strong>backup.zip<\/strong><\/pre>\n<p><strong>UPLOADING AN ARCHIVE LARGER THAN 4 GB<\/strong><\/p>\n<p>Uploading files over 4 GB is strongly discouraged because it requires multipart upload.<\/p>\n<p>Instead, use <code>zip<\/code>, <code>tar<\/code>, <code>split<\/code>, or any similar tool to break the file into smaller chunks, then use the <code>upload-archive<\/code> command shown above.<\/p>\n<p>If multipart upload is unavoidable, start by initiating it and specifying the part size in bytes:<\/p>\n<pre>aws glacier <strong>initiate-multipart-upload<\/strong> --account-id - --part-size <strong>4194304<\/strong> --vault-name <strong>my-glacier<\/strong> --archive-description \"big-file.zip\"<\/pre>\n<p>This will output an <strong>Upload-ID<\/strong> required for all subsequent commands:<\/p>\n<pre>aws glacier upload-multipart-part --body <strong>part-1<\/strong> --range 'bytes <strong>0-4194303<\/strong>\/*' --account-id - --vault-name <strong>my-glacier<\/strong> --upload-id <strong>**upload-id**<\/strong>\r\naws glacier upload-multipart-part --body <strong>part-2<\/strong> --range 'bytes <strong>4194304-8388607<\/strong>\/*' --account-id - --vault-name <strong>my-glacier<\/strong> --upload-id <strong>**upload-id**<\/strong>\r\naws glacier upload-multipart-part --body <strong>part-3<\/strong> --range 'bytes <strong>8388608-12582911<\/strong>\/*' --account-id - --vault-name <strong>my-glacier<\/strong> --upload-id **<strong>upload-id**\r\n<span style=\"color: #ff0000;\">...This process must be scripted as it may need to run hundreds or thousands of times...<\/span><\/strong><\/pre>\n<p><strong>LISTING VAULT CONTENT<\/strong><\/p>\n<p>Submit a job request to get the Job-ID:<\/p>\n<pre>aws glacier <strong>initiate-job<\/strong> --account-id - --vault-name <strong>my-glacier<\/strong> --job-parameters '{\"Type\": \"<strong>inventory-retrieval<\/strong>\"}'<\/pre>\n<p>NOTE: This may take hours or even days. Keep checking whether the status is &#8220;In Progress&#8221; or &#8220;Succeeded&#8221;.<\/p>\n<pre>aws glacier <strong>describe-job<\/strong> --vault-name <strong>my-glacier<\/strong> --account-id - --job-id <strong>tuGTrZxAIwLtvcj6Sv6IU3eoEnuIUX18QeME5x7ENl38UctRykke-jJ9PKQ1YsyVACnkQXd2HLCiWppOcTU2NgUdKjc5<\/strong><\/pre>\n<p>Once succeeded, download the job output (a JSON file):<\/p>\n<pre>aws glacier <strong>get-job-output<\/strong> --vault-name <strong>my-glacier<\/strong> --account-id - --job-id <strong>tuGTrZxAIwLtvcj6Sv6IU3eoEnuIUX18QeME5x7ENl38UctRykke-jJ9PKQ1YsyVACnkQXd2HLCiWppOcTU2NgUdKjc5<\/strong> output.json\r\ncat output.json<\/pre>\n<p>The JSON output, once formatted, will look like this:<\/p>\n<pre>{\r\n   \"VaultARN\":\"arn:aws:glacier:ca-central-1:398812248970:vaults\/my-glacier\",\r\n   \"InventoryDate\":\"2022-02-26T20:39:15Z\",\r\n   \"ArchiveList\":[\r\n      {\r\n         \"ArchiveId\":\"<strong>2qbH3Hgyq0rdnu4-xZQNiNLh9lkODN1orUMD-dJkNuAC6YQASPHAAH8LaEijaoEcYaRGNCjrH1u-zlqHOzZoRUCHr-JOWiqg_PsLpuzSDsb48SaKEtvBYUGZ_tY0jN19OhBlTkM_tA<\/strong>\",\r\n         \"ArchiveDescription\":\"\",\r\n         \"CreationDate\":\"2022-02-26T17:22:53Z\",\r\n         \"Size\":153717283,\r\n         \"SHA256TreeHash\":\"62d22c320d76356dd9aa6032bdd1bca43c3f1aadef085cd7f5f327e6f9e2b004\"\r\n      },\r\n      {\r\n         \"ArchiveId\":\"<strong>lhIYAD3uqPnweEtP4yQII_BhZjy4JBOaprFlmsqe0gJ08V5ccOwdQ3eSxc1NXjXm2BqxQlidiDiaCSgOtuwJQvQpZ-Z0qw7tXRM0QHKUntPYeaowJJlmbvQK4d-fIEMTNjRsVJwKUw<\/strong>\",\r\n         \"ArchiveDescription\":\"\",\r\n         \"CreationDate\":\"2022-02-26T17:26:12Z\",\r\n         \"Size\":646743001,\r\n         \"SHA256TreeHash\":\"70d5a5c0323dc9fc43edacee92f4a88cda5ac7369751dddc246a8b09e619ee74\"\r\n      },\r\n      {\r\n         \"ArchiveId\":\"<strong>TniZ4K9rvxujzzfXI8Mb5Ioz8G_OCHgr0G4LmJHHM9hijmIxSEeGMfaoPW_n3XvHMZsh9gK9rvp04Vkf8Pk7paVHwEKylfSAz7I7YUvpZ7VC7hmGlJ0ZB69EO9rc3RJSuJipABzfRg<\/strong>\",\r\n         \"ArchiveDescription\":\"\",\r\n         \"CreationDate\":\"2022-02-26T17:28:08Z\",\r\n         \"Size\":592047837,\r\n         \"SHA256TreeHash\":\"d3ba9031a1212c495bedbf05c944bf5707011650141421fa6b50234866bec4d3\"\r\n      }\r\n   ]\r\n}<\/pre>\n<p><strong>NOTE:<\/strong> The highlighted values are the Archive-IDs of the uploaded files. These IDs are required to retrieve or delete archives. Be aware that each <code>get-job-output<\/code> link expires after <strong>24 hours<\/strong>.<\/p>\n<p><strong>DELETING AN ARCHIVE<\/strong><\/p>\n<pre>aws glacier <strong>delete-archive<\/strong> --vault-name <strong>my-glacier<\/strong> --account-id - --archive-id <strong>2qbH3Hgyq0rdnu4-xZQNiNLh9lkODN1orUMD-dJkNuAC6YQASPHAAH8LaEijaoEcYaRGNCjrH1u-zlqHOzZoRUCHr-JOWiqg_PsLpuzSDsb48SaKEtvBYUGZ_tY0jN19OhBlTkM_tA<\/strong><\/pre>\n<p><strong>DOWNLOADING AN ARCHIVE<\/strong><\/p>\n<p>First, submit a job request to get the Job-ID:<\/p>\n<pre>aws glacier <strong>initiate-job<\/strong> --account-id - --vault-name <strong>my-glacier<\/strong> --job-parameters '{\"Type\": \"<strong>archive-retrieval<\/strong>\"}'<\/pre>\n<p>Once the job succeeds:<\/p>\n<pre>aws glacier <strong>get-job-output<\/strong> --account-id - --vault-name <strong>my-glacier<\/strong> --job-id <strong>tuGTrZxAIwLtvcj6Sv6IU3eoEnuIUX18QeME5x7ENl38UctRykke-jJ9PKQ1YsyVACnkQXd2HLCiWppOcTU2NgUdKjc5<\/strong> backup.zip<\/pre>\n<hr \/>\n<p><strong>REFLECTIONS<\/strong><\/p>\n<p>This AWS service is not user-friendly and I would only recommend it for personal use in extreme cases, as it requires significant scripting and automation.<\/p>\n<p>Retrieval time and cost are manageable if you never plan on accessing the archives, but the retrieval process itself is painful.<\/p>\n<p>Whenever possible, consider using a regular S3 bucket for the same purpose.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>S3 Glacier is an AWS storage service intended strictly as a last-resort backup, only if [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[7],"tags":[],"class_list":["post-2823","post","type-post","status-publish","format-standard","hentry","category-web"],"_links":{"self":[{"href":"https:\/\/dft.wiki\/index.php?rest_route=\/wp\/v2\/posts\/2823","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dft.wiki\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dft.wiki\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dft.wiki\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/dft.wiki\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=2823"}],"version-history":[{"count":6,"href":"https:\/\/dft.wiki\/index.php?rest_route=\/wp\/v2\/posts\/2823\/revisions"}],"predecessor-version":[{"id":5688,"href":"https:\/\/dft.wiki\/index.php?rest_route=\/wp\/v2\/posts\/2823\/revisions\/5688"}],"wp:attachment":[{"href":"https:\/\/dft.wiki\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=2823"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dft.wiki\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=2823"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dft.wiki\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=2823"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}