- Description:
- Firecracker with the chroot.dev fixes, branch chroot
- Last Change:
- Clone URL:
ssh://anon@got.v6.chroot.run/firecracker
- ED25519:
SHA256:7587EKj+viWIfWGzPGNqUFSdFEh83CZFaqnwaoNmcNc
Commit Briefs
Merge branch 'main' into dependabot/cargo/firecracker-e368cb612c (dependabot/cargo/firecracker-e368cb612c)
vcpu: Complete pending I/O before honoring immediate_exit
Firecracker can acknowledge a pause after handling an MMIO or PIO exit without completing the guest instruction. Commit 32cfc5e8db93 ("skipped unnecessary kvm_immediate_exits") added a userspace immediate_exit shortcut that skips KVM_RUN, but KVM requires re-entry to finish I/O. Snapshots can then omit the pending RAM write or register update because KVM does not expose the unfinished operation in saved vCPU state. Always issue KVM_RUN and let KVM complete pending I/O before returning EINTR for immediate_exit, without executing further guest instructions. Keep the existing EINTR flag clearing so the vCPU can resume normally. The test runs a guest that loads a byte from an MMIO address into a register, on both x86_64 and aarch64. After handling the MMIO exit, it sets immediate_exit and calls KVM_RUN again. It checks that the register holds the MMIO response, but the following guest instruction has not executed. A final KVM_RUN checks that this instruction executes after the flag has been cleared. Fixes: 32cfc5e8db93 ("skipped unnecessary kvm_immediate_exits") Signed-off-by: James Curtis <jxcurtis@amazon.co.uk>
chore(deps): bump the firecracker group across 1 directory with 12 updates
Bumps the firecracker group with 12 updates in the / directory: | Package | From | To | | --- | --- | --- | | [thiserror](https://github.com/dtolnay/thiserror) | `2.0.20` | `2.0.21` | | [zerocopy](https://github.com/google/zerocopy) | `0.8.57` | `0.8.59` | | [cc](https://github.com/rust-lang/cc-rs) | `1.4.7` | `1.5.1` | | [find-msvc-tools](https://github.com/rust-lang/cc-rs) | `0.1.13` | `0.1.14` | | [js-sys](https://github.com/wasm-bindgen/wasm-bindgen) | `0.3.105` | `0.3.106` | | [lazy_static](https://github.com/rust-lang-nursery/lazy-static.rs) | `1.5.0` | `1.5.1` | | [rand](https://github.com/rust-random/rand) | `0.10.2` | `0.10.3` | | [wasm-bindgen](https://github.com/wasm-bindgen/wasm-bindgen) | `0.2.128` | `0.2.129` | | [wasm-bindgen-macro](https://github.com/wasm-bindgen/wasm-bindgen) | `0.2.128` | `0.2.129` | | [wasm-bindgen-macro-support](https://github.com/wasm-bindgen/wasm-bindgen) | `0.2.128` | `0.2.129` | | [wasm-bindgen-shared](https://github.com/wasm-bindgen/wasm-bindgen) | `0.2.128` | `0.2.129` | | [zerocopy-derive](https://github.com/google/zerocopy) | `0.8.57` | `0.8.59` | Updates `thiserror` from 2.0.20 to 2.0.21 - [Release notes](https://github.com/dtolnay/thiserror/releases) - [Commits](https://github.com/dtolnay/thiserror/compare/2.0.20...2.0.21) Updates `zerocopy` from 0.8.57 to 0.8.59 - [Release notes](https://github.com/google/zerocopy/releases) - [Commits](https://github.com/google/zerocopy/compare/v0.8.57...v0.8.59) Updates `cc` from 1.4.7 to 1.5.1 - [Release notes](https://github.com/rust-lang/cc-rs/releases) - [Changelog](https://github.com/rust-lang/cc-rs/blob/main/CHANGELOG.md) - [Commits](https://github.com/rust-lang/cc-rs/compare/cc-v1.4.7...cc-v1.5.1) Updates `find-msvc-tools` from 0.1.13 to 0.1.14 - [Release notes](https://github.com/rust-lang/cc-rs/releases) - [Changelog](https://github.com/rust-lang/cc-rs/blob/main/CHANGELOG.md) - [Commits](https://github.com/rust-lang/cc-rs/compare/find-msvc-tools-v0.1.13...find-msvc-tools-v0.1.14) Updates `js-sys` from 0.3.105 to 0.3.106 - [Release notes](https://github.com/wasm-bindgen/wasm-bindgen/releases) - [Changelog](https://github.com/wasm-bindgen/wasm-bindgen/blob/main/CHANGELOG.md) - [Commits](https://github.com/wasm-bindgen/wasm-bindgen/commits) Updates `lazy_static` from 1.5.0 to 1.5.1 - [Release notes](https://github.com/rust-lang-nursery/lazy-static.rs/releases) - [Commits](https://github.com/rust-lang-nursery/lazy-static.rs/commits) Updates `rand` from 0.10.2 to 0.10.3 - [Release notes](https://github.com/rust-random/rand/releases) - [Changelog](https://github.com/rust-random/rand/blob/master/CHANGELOG.md) - [Commits](https://github.com/rust-random/rand/compare/0.10.2...0.10.3) Updates `wasm-bindgen` from 0.2.128 to 0.2.129 - [Release notes](https://github.com/wasm-bindgen/wasm-bindgen/releases) - [Changelog](https://github.com/wasm-bindgen/wasm-bindgen/blob/main/CHANGELOG.md) - [Commits](https://github.com/wasm-bindgen/wasm-bindgen/compare/0.2.128...0.2.129) Updates `wasm-bindgen-macro` from 0.2.128 to 0.2.129 - [Release notes](https://github.com/wasm-bindgen/wasm-bindgen/releases) - [Changelog](https://github.com/wasm-bindgen/wasm-bindgen/blob/main/CHANGELOG.md) - [Commits](https://github.com/wasm-bindgen/wasm-bindgen/compare/0.2.128...0.2.129) Updates `wasm-bindgen-macro-support` from 0.2.128 to 0.2.129 - [Release notes](https://github.com/wasm-bindgen/wasm-bindgen/releases) - [Changelog](https://github.com/wasm-bindgen/wasm-bindgen/blob/main/CHANGELOG.md) - [Commits](https://github.com/wasm-bindgen/wasm-bindgen/compare/0.2.128...0.2.129) Updates `wasm-bindgen-shared` from 0.2.128 to 0.2.129 - [Release notes](https://github.com/wasm-bindgen/wasm-bindgen/releases) - [Changelog](https://github.com/wasm-bindgen/wasm-bindgen/blob/main/CHANGELOG.md) - [Commits](https://github.com/wasm-bindgen/wasm-bindgen/compare/0.2.128...0.2.129) Updates `zerocopy-derive` from 0.8.57 to 0.8.59 - [Release notes](https://github.com/google/zerocopy/releases) - [Commits](https://github.com/google/zerocopy/compare/v0.8.57...v0.8.59) --- updated-dependencies: - dependency-name: thiserror dependency-version: 2.0.21 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: firecracker - dependency-name: zerocopy dependency-version: 0.8.59 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: firecracker - dependency-name: cc dependency-version: 1.5.1 dependency-type: indirect update-type: version-update:semver-minor dependency-group: firecracker - dependency-name: find-msvc-tools dependency-version: 0.1.14 dependency-type: indirect update-type: version-update:semver-patch dependency-group: firecracker - dependency-name: js-sys dependency-version: 0.3.106 dependency-type: indirect update-type: version-update:semver-patch dependency-group: firecracker - dependency-name: lazy_static dependency-version: 1.5.1 dependency-type: indirect update-type: version-update:semver-patch dependency-group: firecracker - dependency-name: rand dependency-version: 0.10.3 dependency-type: indirect update-type: version-update:semver-patch dependency-group: firecracker - dependency-name: wasm-bindgen dependency-version: 0.2.129 dependency-type: indirect update-type: version-update:semver-patch dependency-group: firecracker - dependency-name: wasm-bindgen-macro dependency-version: 0.2.129 dependency-type: indirect update-type: version-update:semver-patch dependency-group: firecracker - dependency-name: wasm-bindgen-macro-support dependency-version: 0.2.129 dependency-type: indirect update-type: version-update:semver-patch dependency-group: firecracker - dependency-name: wasm-bindgen-shared dependency-version: 0.2.129 dependency-type: indirect update-type: version-update:semver-patch dependency-group: firecracker - dependency-name: zerocopy-derive dependency-version: 0.8.59 dependency-type: indirect update-type: version-update:semver-patch dependency-group: firecracker ... Signed-off-by: dependabot[bot] <support@github.com>
docs(changelog): Note guest memory discard changes
Record that balloon and virtio-mem discards now free shared memfd backing, and that on hugetlbfs they free whole huge pages and zero partial ones, which can increase host memory use with the traditional balloon. Signed-off-by: Jay Chung <jaehoc@amazon.com>
docs: Describe memfd and huge page discard
Describe discard for anonymous, shared memfd and private snapshot memory, and present vhost-user block as an example of shared memfd backing. State that discarded ranges read as zero. For hugetlbfs, warn against the traditional balloon. Only the whole huge pages inside a range are freed and the partial ones are zeroed, so traditional inflation frees no hugetlbfs memory. It can also increase host memory use, because the guest can inflate pages it never touched. Recommend free page hinting or reporting instead, also in the balloon caveats, and state that reporting needs the guest to report ranges of at least 2 MiB, also in the reporting section. Preserve the explanation of RSS reduction for reporting and hinting, and note that hugetlbfs memory is accounted separately. In the hugepages doc, explain why the traditional balloon cannot free hugetlbfs memory and how inflation can increase host memory use, and note that freeing anonymous hugetlbfs memory needs a 5.18 or later host, since 5.10 rejects MADV_DONTNEED on hugetlbfs. Tell page fault handler authors that zeroed partial huge pages send no UFFD_EVENT_REMOVE and can arrive as page faults, and mention MADV_REMOVE for unplugged memfd memory in the memory hotplug doc. Signed-off-by: Jay Chung <jaehoc@amazon.com>
test: Measure memfd backing in resident memory
RSS counts the pages mapped into the process, not the pages a memfd holds. MADV_DONTNEED on a memfd mapping lowers RSS while the memfd keeps its pages, so an RSS check cannot tell whether memfd-backed guest memory was freed. When Firecracker holds a guest memory memfd, make get_resident_memory count its private RSS plus the blocks allocated to the memfd. Run test_size_reduction on memfd-backed guest memory as well, through a vhost-user root drive. Its huge page variants do not need hugetlb MADV_DONTNEED support on the host, because discarding shared memfd memory uses MADV_REMOVE. Signed-off-by: Jay Chung <jaehoc@amazon.com>
feat(memory): Support shared memfd discard
MADV_DONTNEED on a shared file mapping only drops the page table entries. The memfd keeps its pages, so balloon and virtio-mem discards on memory shared with vhost-user devices free nothing. Use MADV_REMOVE for shared file mappings. The kernel frees the backing pages at the mapping's file offset and still sends UFFD_EVENT_REMOVE, which fallocate on the memfd would bypass. Hugetlbfs memfds only see whole huge pages here, since discard_range zeroes the partial ones. Test discards at nonzero file offsets and on hugetlbfs memfds, shared writes after a discard, and the userfaultfd remove notification in an integration test. Closes #6167 Signed-off-by: Jay Chung <jaehoc@amazon.com>
fix(memory): Discard whole huge pages, zero edges
A huge page can only be freed whole. For a range that covers part of one, MADV_DONTNEED on anonymous hugetlb memory either rejects the range or rounds its end down, depending on the kernel. In the second case the call succeeds while the partial page keeps its old contents. Discard only the whole backing pages inside the range, and write zeros to the partial pages at either end, so that a discarded range always reads as zero. The write goes through the mapping, so under userfaultfd the edges fault through the handler like any other guest write. Edges in unplugged hotplug slots are skipped: those slots are PROT_NONE, and snapshots store them as holes. Writing allocates a partial huge page that was not backed yet. With the guest's default settings only traditional inflation produces partial ranges. Linux guests inflate at most 1 MiB at a time, so it never frees hugetlbfs memory, and now it can increase host memory use. Explain this where discard_range splits the range. Signed-off-by: Jay Chung <jaehoc@amazon.com>
refactor(memory): Split discard_range by mapping
Give private file, shared file and anonymous mappings their own match arms in discard_range. Move the private file remap into remap_anonymous_range, and the madvise call and its error handling into madvise_range. Shared file mappings still use MADV_DONTNEED. Move the page size lookup out of RawGuestRegionMmap::new into page_size, so that discard_range can use it. Also reject ranges that run past the end of the region. The only caller already clamps each range to its region, so nothing changes today, but the SAFETY comment can now rely on a check. Signed-off-by: Jay Chung <jaehoc@amazon.com>
docs(mmds): Note TX rate limiter skips MMDS frames
Frames the guest sends to MMDS are handled inside Firecracker and never reach the TAP device, so the net device refunds their TX rate limiter tokens. The docs did not say so, and an operator setting a TX rate limiter on an MMDS-enabled interface could expect it to bound all guest traffic on that interface. Add a note to the MMDS security considerations saying the TX rate limiter does not cover MMDS traffic, and pointing to the per-microVM CPU quota for bounding the time Firecracker spends processing it. Format it and the existing packet filtering note as GitHub NOTE alerts, so both stand out and read as separate items. Signed-off-by: Jay Chung <jaehoc@amazon.com>
Branches
Tree
README.md
<picture>
<source media="(prefers-color-scheme: dark)" srcset="docs/images/fc_logo_full_transparent-bg_white-fg.png">
<source media="(prefers-color-scheme: light)" srcset="docs/images/fc_logo_full_transparent-bg.png">
<img alt="Firecracker Logo Title" width="750" src="docs/images/fc_logo_full_transparent-bg.png">
</picture>
Our mission is to enable secure, multi-tenant, minimal-overhead execution of
container and function workloads.
Read more about the Firecracker Charter [here](CHARTER.md).
## What is Firecracker?
Firecracker is an open source virtualization technology that is purpose-built
for creating and managing secure, multi-tenant container and function-based
services that provide serverless operational models. Firecracker runs workloads
in lightweight virtual machines, called microVMs, which combine the security and
isolation properties provided by hardware virtualization technology with the
speed and flexibility of containers.
## Overview
The main component of Firecracker is a virtual machine monitor (VMM) that uses
the Linux Kernel Virtual Machine (KVM) to create and run microVMs. Firecracker
has a minimalist design. It excludes unnecessary devices and guest-facing
functionality to reduce the memory footprint and attack surface area of each
microVM. This improves security, decreases the startup time, and increases
hardware utilization. Firecracker has also been integrated in container
runtimes, for example
[Kata Containers](https://github.com/kata-containers/kata-containers) and
[Flintlock](https://github.com/liquidmetal-dev/flintlock).
Firecracker was developed at Amazon Web Services to accelerate the speed and
efficiency of services like [AWS Lambda](https://aws.amazon.com/lambda/) and
[AWS Fargate](https://aws.amazon.com/fargate/). Firecracker is open sourced
under [Apache version 2.0](LICENSE).
To read more about Firecracker, check out
[firecracker-microvm.io](https://firecracker-microvm.github.io).
## Getting Started
To get started with Firecracker, download the latest
[release](https://github.com/firecracker-microvm/firecracker/releases) binaries
or build it from source.
You can build Firecracker on any Unix/Linux system that has Docker running (we
use a development container) and `bash` installed, as follows:
```bash
git clone https://github.com/firecracker-microvm/firecracker
cd firecracker
tools/devtool build
toolchain="$(uname -m)-unknown-linux-musl"
```
The Firecracker binary will be placed at
`build/cargo_target/${toolchain}/debug/firecracker`. For more information on
building, testing, and running Firecracker, go to the
[quickstart guide](docs/getting-started.md).
The overall security of Firecracker microVMs, including the ability to meet the
criteria for safe multi-tenant computing, depends on a well configured Linux
host operating system. A configuration that we believe meets this bar is
included in [the production host setup document](docs/prod-host-setup.md).
## Contributing
Firecracker is already running production workloads within AWS, but it's still
Day 1 on the journey guided by our [mission](CHARTER.md). There's a lot more to
build and we welcome all contributions.
To contribute to Firecracker, check out the development setup section in the
[getting started guide](docs/getting-started.md) and then the Firecracker
[contribution guidelines](CONTRIBUTING.md).
## Releases
New Firecracker versions are released via the GitHub repository
[releases](https://github.com/firecracker-microvm/firecracker/releases) page,
typically every two or three months. A history of changes is recorded in our
[changelog](CHANGELOG.md).
The Firecracker release policy is detailed [here](docs/RELEASE_POLICY.md).
## Design
Firecracker's overall architecture is described in
[the design document](docs/design.md).
## Features & Capabilities
Firecracker consists of a single micro Virtual Machine Manager process that
exposes an API endpoint to the host once started. The API is
[specified in OpenAPI format](src/firecracker/swagger/firecracker.yaml). Read
more about it in the [API docs](docs/api_requests).
The **API endpoint** can be used to:
- Configure the microvm by:
- Setting the number of vCPUs (the default is 1).
- Setting the memory size (the default is 128 MiB).
- Configuring a [CPU template](docs/cpu_templates/cpu-templates.md).
- Add one or more network interfaces to the microVM.
- Add one or more read-write or read-only disks to the microVM, each represented
by a file-backed block device.
- Trigger a block device re-scan while the guest is running. This enables the
guest OS to pick up size changes to the block device's backing file.
- Change the backing file for a block device, before or after the guest boots.
- Configure rate limiters for virtio devices which can limit the bandwidth,
operations per second, or both.
- Configure the logging and metric system.
- `[BETA]` Configure the data tree of the guest-facing metadata service. The
service is only available to the guest if this resource is configured.
- Add a [vsock socket](docs/vsock.md) to the microVM.
- Add a [entropy device](docs/entropy.md) to the microVM.
- Add a [pmem device](docs/pmem.md) to the microVM.
- Configure and manage [memory hotplugging](docs/memory-hotplug.md).
- `[Developer Preview]` [Hot-plug and hot-unplug](docs/device-hotplug.md) virtio
PCI devices while the VM is running.
- Start the microVM using a given kernel image, root file system, and boot
arguments.
- [x86_64 only] Stop the microVM.
**Built-in Capabilities**:
- Demand fault paging and CPU oversubscription enabled by default.
- Advanced, thread-specific seccomp filters for enhanced security.
- [Jailer](docs/jailer.md) process for starting Firecracker in production
scenarios; applies a cgroup/namespace isolation barrier and then drops
privileges.
## Tested platforms
We test all combinations of:
| Instance | Host OS & Kernel | Guest Rootfs | Guest Kernel |
| :------------------------------------------ | :---------------- | :----------- | :----------- |
| m5n.metal (Intel Cascade Lake) | al2 linux_5.10 | ubuntu 24.04 | linux_5.10 |
| m6i.metal (Intel Ice Lake) | al2023 linux_6.1 | | linux_6.1 |
| | al2023 linux_6.18 | | |
| m7i.metal-24xl (Intel Sapphire Rapids) | | | |
| m7i.metal-48xl (Intel Sapphire Rapids) | | | |
| **m8i.metal-48xl (Intel Granite Rapids)\*** | | | |
| **m8i.metal-96xl (Intel Granite Rapids)\*** | | | |
| m6a.metal (AMD Milan) | | | |
| m7a.metal-48xl (AMD Genoa) | | | |
| m6g.metal (Graviton 2) | | | |
| m7g.metal (Graviton 3) | | | |
| m8g.metal-24xl (Graviton 4) | | | |
| m8g.metal-48xl (Graviton 4) | | | |
| m9g.metal-48xl (Graviton 5) | | | |
**\***: We **only** support AWS EC2 8th Gen Intel (\*8i) instances using a 6.1
or 6.18 host kernel. This is due to poor kernel support for Granite Rapids CPUs
on 5.10.
## Known issues and Limitations
- The `pl031` RTC device on aarch64 does not support interrupts, so guest
programs which use an RTC alarm (e.g. `hwclock`) will not work.
## Performance
Firecracker's performance characteristics are listed as part of the
[specification documentation](SPECIFICATION.md). All specifications are a part
of our commitment to supporting container and function workloads in serverless
operational models, and are therefore enforced via continuous integration
testing.
## Policy for Security Disclosures
The security of Firecracker is our top priority. If you suspect you have
uncovered a vulnerability, contact us privately, as outlined in our
[security policy document](SECURITY.md); we will immediately prioritize your
disclosure.
## FAQ & Contact
Frequently asked questions are collected in our [FAQ doc](FAQ.md).
You can get in touch with the Firecracker community in the following ways:
- Security-related issues, see our [security policy document](SECURITY.md).
- Chat with us on our
[Slack workspace](https://join.slack.com/t/firecracker-microvm/shared_invite/zt-3v81btcpe-usCf8Qk7k1gUlSAEKKdYMg)
_Note: most of the maintainers are on a European time zone._
- Open a GitHub issue in this repository.
- Email the maintainers at
[firecracker-maintainers@amazon.com](mailto:firecracker-maintainers@amazon.com).
When communicating within the Firecracker community, please mind our
[code of conduct](CODE_OF_CONDUCT.md).