- Description:
- Firecracker with the chroot.dev fixes, branch chroot
- Last Change:
- Clone URL:
ssh://anon@got.v6.chroot.run/firecracker
- ED25519:
SHA256:7587EKj+viWIfWGzPGNqUFSdFEh83CZFaqnwaoNmcNc
Commit Briefs
chore: Release v1.16.2 (firecracker-v1.16, tags/v1.16.2)
Update version number / CHANGELOG / CREDITS Signed-off-by: Ilias Stamatis <ilstam@amazon.com>
docs: Update RELEASE_POLICY with v1.16.2
v1.16.2 is the latest patch for the v1.16 release series. Signed-off-by: Ilias Stamatis <ilstam@amazon.com>
docs(changelog): note the virtio-mem block state fix
Record that virtio-mem now commits a slot's block state only after its KVM update succeeds, keeping the device's bookkeeping consistent with KVM when a plug or unplug request fails part way through. (cherry picked from commit 9384f395f5d47f05b68a657525e209f8aa9c8264) Signed-off-by: Jay Chung <jaehoc@amazon.com>
fix(vmm): commit virtio-mem block state after the KVM slot update
update_range committed the block state (plugged_blocks, plugged_size) for the whole request up front and only then updated the KVM slots, and update_slot flipped its per-slot bitmap before issuing the KVM ioctl. On a multi-slot request that failed part way through, the committed state covered the whole range while KVM reflected only the slots that had been updated, leaving the device's bookkeeping inconsistent with KVM. Update KVM one slot at a time and commit that slot's block state only after its KVM update succeeds; discard an unplugged slot only once it is committed. A partial failure now stops with the block state and the KVM slots reflecting exactly the slots that were updated, and never discards a block that has not been committed as unplugged. In update_slot, commit the slot bitmap only once both the protection change and the KVM slot update have succeeded, and roll back the first of the two when the second fails, so a failed slot update leaves the slot exactly as it was. A failed rollback would leave the host mapping and KVM out of sync with no way back, so it panics. The whole-range discard is now done per slot as each slot commits. This issues one discard per intersecting slot instead of one for the range, which is not on a hot path. Add a test that plugs a range spanning two KVM slots with the second slot's memory-region update forced to fail, and checks that only the first slot's block is left committed. A test-only fault injector on set_user_memory_region drives the failure. (cherry picked from commit 1e99e529f7ae1ee0ffbeaba764552374fb71f5ea) [backport: depends on the KVM-first ordering from #5944, picked as the previous commit; resolved a conflict with 1.16's discard block in update_range, the result matches main] Signed-off-by: Jay Chung <jaehoc@amazon.com>
fix: virtio-mem: update kvm slots before discarding memory
In `update_range` function, move kvm slot update before memory discard to prevent a small time window when the guest can re-fault just discarded memory before it is removed from kvm view. (cherry picked from commit 8c4bc5a522cd21a80fc94396194b0d33e685bd85) [backport: prerequisite for the #6176 backport that follows] Signed-off-by: Egor Lazarchuk <yegorlz@amazon.co.uk>
test: x86: update T2CL MSR baselines
Refresh the T2CL baselines for Cascade Lake and Ice Lake after Firecracker sets each vCPU's CPUID before retrieving MSRs to be modified by CPU templates. The corrected output exposes IA32_SPEC_CTRL and IA32_TSX_CTRL and records the post-CPUID IA32_ARCH_CAPABILITIES values instead of zero across guest kernels 5.10, 6.1, and 6.18. For each CPU model, all three MSR results match the corresponding Linux 6.1 host baselines. This confirms that the ordering fix restores the established T2CL behavior on Linux 6.18 hosts. [Dropped the two *_6.18host_6.18guest.csv baseline updates, since 6.18 guest kernels are not supported on this branch.] [cherry picked from commit 9c493b4dadc470d34746b73d226e568180234ee3] Signed-off-by: Takahiro Itazuri <itazur@amazon.com>
tests: x86: verify CPUID-gated MSR passthrough
Add an integration test that verifies CPU templates preserve passthrough bits from a CPUID-gated MSR. Compare a live IA32_ARCH_CAPABILITIES value from a no-template guest against a custom template that flips bit 0 and passes all other bits through. On Linux 6.13 and later, KVM returns zero when userspace reads IA32_ARCH_CAPABILITIES before guest CPUID advertises it [1]. The test therefore covers the requirement that Firecracker install guest CPUID before reading the base MSR values used for template application. [1]: https://github.com/torvalds/linux/commit/a5d563890b8f0352c8f915c6acc75b5cd3b28d98 [The test fixture consolidation is not backported, so pin_guest_kernel() and GUEST_KERNEL_DEFAULT do not exist here and the test takes the guest_kernel_default fixture instead. Also, framework.utils_cpu_templates only provides get_cpu_template_name on this branch.] [cherry picked from commit 4aa9bb83c1c131ec3b437dea51ac035f7e0647a9] Signed-off-by: Takahiro Itazuri <itazur@amazon.com>
vmm: x86: remove obsolete CPU configuration path
Remove the CpuConfiguration and KvmVcpu compatibility wrappers left after wiring staged CPUID and MSR configuration into the x86 cold boot path. Update x86 tests to exercise the staged operations directly. CpuConfiguration remains available for CPU configuration dumps, and VcpuConfig remains available for aarch64 boot configuration. [cherry picked from commit d9edec6f4646dbed9fe10026303a6476181b0f21] Signed-off-by: Takahiro Itazuri <itazur@amazon.com>
vmm: x86: fix CPUID-dependent MSR read ordering
KVM's feature MSR series [1] made userspace accesses to CPUID-dependent MSRs subject to architectural existence checks. Userspace must install guest CPUID with KVM_SET_CPUID{,2} before reading such MSRs with KVM_GET_MSRS. If guest CPUID does not advertise a feature, KVM returns zero when userspace reads the corresponding MSR that KVM advertised as supported. The KVM maintainer deferred documenting this behavior until KVM's internal MSR handling is fully converted to KVM_MSR_RET_UNSUPPORTED [3]. The documentation update has not landed as of Linux 6.18. This behavior landed in Linux 6.13 through commit [2], and is therefore present in Firecracker's supported 6.18 host kernel. Firecracker read CPU template MSRs from vCPU 0 before installing guest CPUID. On T2CL, KVM_GET_MSRS consequently returned zero for IA32_ARCH_CAPABILITIES. The template's passthrough mask preserved the zero IBRS_ALL bit, and the later KVM_SET_MSRS overwrote KVM's post-CPUID value. The guest did not observe eIBRS support and selected retpoline. Correct CPU configuration ordering as follows: 1. Apply CPUID modifiers. 2. Install each vCPU's normalized CPUID with KVM_SET_CPUID2. 3. Retrieve CPUID-dependent MSRs from vCPU 0 with KVM_GET_MSRS. 4. Apply MSR modifiers. 5. Add Linux boot MSRs and update CPUID-derived MSR snapshot bookkeeping. 6. Install each vCPU's MSRs with KVM_SET_MSRS. [1]: https://lore.kernel.org/all/20240802185511.305849-1-seanjc@google.com/ [2]: https://github.com/torvalds/linux/commit/a5d563890b8f0352c8f915c6acc75b5cd3b28d98 [3]: https://lore.kernel.org/kvm/Zivh0IaAmHsEOLFc@google.com/ [cherry picked from commit 4681a67f712a9fc7cb2483cf2884d4144ad3cf4e] Signed-off-by: Takahiro Itazuri <itazur@amazon.com>
vmm: x86: vcpu: split boot configuration stages
Extract CPUID setup, MSR setup, and the remaining Linux boot state from KvmVcpu::configure(), and keep configure() as a compatibility wrapper. This is a preparatory refactor for a follow-up fix. The fix needs to install CPUID on every vCPU before reading CPUID-dependent MSRs, such as IA32_ARCH_CAPABILITIES, from vCPU 0. Exposing the operations independently lets the follow-up commit correct the ordering. Return the installed KVM CPUID from configure_cpuid() so MSR snapshot bookkeeping continues to use the exact per-vCPU CPUID and preserves the existing MSR save-list order. No functional change is intended. [cherry picked from commit b851857505c7b6ea78b04f22296e63cbf6b9c73f] Signed-off-by: Takahiro Itazuri <itazur@amazon.com>
Branches
Tree
README.md
<picture>
<source media="(prefers-color-scheme: dark)" srcset="docs/images/fc_logo_full_transparent-bg_white-fg.png">
<source media="(prefers-color-scheme: light)" srcset="docs/images/fc_logo_full_transparent-bg.png">
<img alt="Firecracker Logo Title" width="750" src="docs/images/fc_logo_full_transparent-bg.png">
</picture>
Our mission is to enable secure, multi-tenant, minimal-overhead execution of
container and function workloads.
Read more about the Firecracker Charter [here](CHARTER.md).
## What is Firecracker?
Firecracker is an open source virtualization technology that is purpose-built
for creating and managing secure, multi-tenant container and function-based
services that provide serverless operational models. Firecracker runs workloads
in lightweight virtual machines, called microVMs, which combine the security and
isolation properties provided by hardware virtualization technology with the
speed and flexibility of containers.
## Overview
The main component of Firecracker is a virtual machine monitor (VMM) that uses
the Linux Kernel Virtual Machine (KVM) to create and run microVMs. Firecracker
has a minimalist design. It excludes unnecessary devices and guest-facing
functionality to reduce the memory footprint and attack surface area of each
microVM. This improves security, decreases the startup time, and increases
hardware utilization. Firecracker has also been integrated in container
runtimes, for example
[Kata Containers](https://github.com/kata-containers/kata-containers) and
[Flintlock](https://github.com/liquidmetal-dev/flintlock).
Firecracker was developed at Amazon Web Services to accelerate the speed and
efficiency of services like [AWS Lambda](https://aws.amazon.com/lambda/) and
[AWS Fargate](https://aws.amazon.com/fargate/). Firecracker is open sourced
under [Apache version 2.0](LICENSE).
To read more about Firecracker, check out
[firecracker-microvm.io](https://firecracker-microvm.github.io).
## Getting Started
To get started with Firecracker, download the latest
[release](https://github.com/firecracker-microvm/firecracker/releases) binaries
or build it from source.
You can build Firecracker on any Unix/Linux system that has Docker running (we
use a development container) and `bash` installed, as follows:
```bash
git clone https://github.com/firecracker-microvm/firecracker
cd firecracker
tools/devtool build
toolchain="$(uname -m)-unknown-linux-musl"
```
The Firecracker binary will be placed at
`build/cargo_target/${toolchain}/debug/firecracker`. For more information on
building, testing, and running Firecracker, go to the
[quickstart guide](docs/getting-started.md).
The overall security of Firecracker microVMs, including the ability to meet the
criteria for safe multi-tenant computing, depends on a well configured Linux
host operating system. A configuration that we believe meets this bar is
included in [the production host setup document](docs/prod-host-setup.md).
## Contributing
Firecracker is already running production workloads within AWS, but it's still
Day 1 on the journey guided by our [mission](CHARTER.md). There's a lot more to
build and we welcome all contributions.
To contribute to Firecracker, check out the development setup section in the
[getting started guide](docs/getting-started.md) and then the Firecracker
[contribution guidelines](CONTRIBUTING.md).
## Releases
New Firecracker versions are released via the GitHub repository
[releases](https://github.com/firecracker-microvm/firecracker/releases) page,
typically every two or three months. A history of changes is recorded in our
[changelog](CHANGELOG.md).
The Firecracker release policy is detailed [here](docs/RELEASE_POLICY.md).
## Design
Firecracker's overall architecture is described in
[the design document](docs/design.md).
## Features & Capabilities
Firecracker consists of a single micro Virtual Machine Manager process that
exposes an API endpoint to the host once started. The API is
[specified in OpenAPI format](src/firecracker/swagger/firecracker.yaml). Read
more about it in the [API docs](docs/api_requests).
The **API endpoint** can be used to:
- Configure the microvm by:
- Setting the number of vCPUs (the default is 1).
- Setting the memory size (the default is 128 MiB).
- Configuring a [CPU template](docs/cpu_templates/cpu-templates.md).
- Add one or more network interfaces to the microVM.
- Add one or more read-write or read-only disks to the microVM, each represented
by a file-backed block device.
- Trigger a block device re-scan while the guest is running. This enables the
guest OS to pick up size changes to the block device's backing file.
- Change the backing file for a block device, before or after the guest boots.
- Configure rate limiters for virtio devices which can limit the bandwidth,
operations per second, or both.
- Configure the logging and metric system.
- `[BETA]` Configure the data tree of the guest-facing metadata service. The
service is only available to the guest if this resource is configured.
- Add a [vsock socket](docs/vsock.md) to the microVM.
- Add a [entropy device](docs/entropy.md) to the microVM.
- Add a [pmem device](docs/pmem.md) to the microVM.
- Configure and manage [memory hotplugging](docs/memory-hotplug.md).
- `[Developer Preview]` [Hot-plug and hot-unplug](docs/device-hotplug.md) virtio
PCI devices while the VM is running.
- Start the microVM using a given kernel image, root file system, and boot
arguments.
- [x86_64 only] Stop the microVM.
**Built-in Capabilities**:
- Demand fault paging and CPU oversubscription enabled by default.
- Advanced, thread-specific seccomp filters for enhanced security.
- [Jailer](docs/jailer.md) process for starting Firecracker in production
scenarios; applies a cgroup/namespace isolation barrier and then drops
privileges.
## Tested platforms
We test all combinations of:
| Instance | Host OS & Kernel | Guest Rootfs | Guest Kernel |
| :------------------------------------------ | :---------------- | :----------- | :----------- |
| m5n.metal (Intel Cascade Lake) | al2 linux_5.10 | ubuntu 24.04 | linux_5.10 |
| m6i.metal (Intel Ice Lake) | al2023 linux_6.1 | | linux_6.1 |
| | al2023 linux_6.18 | | |
| m7i.metal-24xl (Intel Sapphire Rapids) | | | |
| m7i.metal-48xl (Intel Sapphire Rapids) | | | |
| **m8i.metal-48xl (Intel Granite Rapids)\*** | | | |
| **m8i.metal-96xl (Intel Granite Rapids)\*** | | | |
| m6a.metal (AMD Milan) | | | |
| m7a.metal-48xl (AMD Genoa) | | | |
| m6g.metal (Graviton 2) | | | |
| m7g.metal (Graviton 3) | | | |
| m8g.metal-24xl (Graviton 4) | | | |
| m8g.metal-48xl (Graviton 4) | | | |
**\***: We **only** support AWS EC2 8th Gen Intel (\*8i) instances using a 6.1
or 6.18 host kernel. This is due to poor kernel support for Granite Rapids CPUs
on 5.10.
## Known issues and Limitations
- The `pl031` RTC device on aarch64 does not support interrupts, so guest
programs which use an RTC alarm (e.g. `hwclock`) will not work.
## Performance
Firecracker's performance characteristics are listed as part of the
[specification documentation](SPECIFICATION.md). All specifications are a part
of our commitment to supporting container and function workloads in serverless
operational models, and are therefore enforced via continuous integration
testing.
## Policy for Security Disclosures
The security of Firecracker is our top priority. If you suspect you have
uncovered a vulnerability, contact us privately, as outlined in our
[security policy document](SECURITY.md); we will immediately prioritize your
disclosure.
## FAQ & Contact
Frequently asked questions are collected in our [FAQ doc](FAQ.md).
You can get in touch with the Firecracker community in the following ways:
- Security-related issues, see our [security policy document](SECURITY.md).
- Chat with us on our
[Slack workspace](https://join.slack.com/t/firecracker-microvm/shared_invite/zt-2tc0mfxpc-tU~HYAYSzLDl5XGGJU3YIg)
_Note: most of the maintainers are on a European time zone._
- Open a GitHub issue in this repository.
- Email the maintainers at
[firecracker-maintainers@amazon.com](mailto:firecracker-maintainers@amazon.com).
When communicating within the Firecracker community, please mind our
[code of conduct](CODE_OF_CONDUCT.md).