Compare commits

...

12 Commits

Author SHA1 Message Date
Maofeng 2650b8ef14 docs(cli): add Wasmeld command reference
Document Component project requirements, live preview behavior and options, package construction, WIT dependency locking and replacement, Registry publishing, and CLI environment variables.

Link the root and Console guides to the maintained CLI reference and remove links to excluded design documents.
2026-07-29 20:52:32 +08:00
Maofeng c982753eb8 docs: clarify live WIT dependency updates
Recommend unlocked development mode when editing local WIT replacements, while retaining --locked for source-only work that must preserve the existing contract graph.
2026-07-29 20:26:57 +08:00
Maofeng 2c6e761288 docs: document live Component preview workflow
Explain the wasmeld dev watch-build-package-deploy loop, isolated development service IDs, hash revisions, failure retention, cleanup behavior, and command options.

Document the inactive-revision DELETE endpoint and the switch-before-unregister safety rule.
2026-07-29 20:26:25 +08:00
Maofeng 839874fb56 feat(cli): add live Component development loop
Add wasmeld dev with source and local WIT replace watching, Debug builds by default, deterministic hash-based development revisions, and optional one-shot or Release operation.

Build and validate before registering, switch the isolated development Deployment only after successful startup, and unregister the prior matching development revision after the switch.

Keep generated WIT dependencies and build output outside the watch fingerprint, retain the previous deployment after failures, and avoid introducing a platform-specific watcher dependency.
2026-07-29 20:26:16 +08:00
Maofeng 201d99c7e0 feat(console-ui): display revision unregister events
Extend the typed event model and activity feed metadata so development cleanup remains visible in the management console.
2026-07-29 20:26:10 +08:00
Maofeng d0d2e06f68 feat(console): unregister inactive revisions
Expose a management DELETE endpoint that refuses to remove an active Deployment, delegates Actor and compiled Component cleanup to Runtime, and removes stored artifacts.

Reconcile Toasty snapshots by deleting absent service and Deployment rows so removed revisions do not reappear after restart.

Persist unregister lifecycle events and verify active-revision protection, artifact deletion, Runtime cleanup, and restart behavior.
2026-07-29 20:26:06 +08:00
Maofeng 778fbd2a06 feat(runtime): unregister service revisions
Add a lifecycle-serialized unregister operation that stops a resident Actor before releasing its registered and compiled Component.

Return the existing not-registered and unavailable errors without leaving a partially removed live revision, and cover Actor and registration count cleanup.
2026-07-29 20:25:58 +08:00
Maofeng 71654a8bcf docs: document persistent Host KV
Describe the versioned KV capability, service-level namespace, persistence behavior, limits, and kv-probe packaging workflow.

Record the v0.1 exclusions for TTL, enumeration, transactions, and compare-and-swap.
2026-07-29 19:37:19 +08:00
Maofeng a325c60421 feat(console): persist Host KV in libSQL
Store KV entries through Toasty with a composite service ID and key primary key, including an additive table migration for existing Console databases.

Bridge synchronous Wasmtime Host calls to async persistence with a bounded worker queue and response timeout capped by the service deadline.

Verify same-service revision sharing, cross-service isolation, deletion, exact capability reporting, and persistence after reopening Console.
2026-07-29 19:37:17 +08:00
Maofeng 8d8d136510 feat(runtime): host service-scoped KV capability
Introduce an injectable, deadline-aware KvBackend boundary and link only the exact wasmeld:kv/store@0.1.0 import requested by a Component.

Namespace entries by service ID so revisions share state while services remain isolated, reject registrations without a backend, and enforce 256-byte keys and 64 KiB values.

Cover capability discovery, cross-revision sharing, service isolation, missing backends, and resource limits with Component integration tests.
2026-07-29 19:37:10 +08:00
Maofeng 044b64a9c7 feat(component): add KV capability probe
Add a small Component that imports wasmeld:kv/store@0.1.0 and exercises get, set, and delete through a byte-oriented test protocol.

Pin service and KV WIT dependencies in wit.lock while keeping generated wit/deps content out of source control.
2026-07-29 19:37:05 +08:00
Maofeng 5edf185c90 feat(wit): add versioned KV store package
Define wasmeld:kv/store@0.1.0 with get, set, and delete operations.

Expose explicit invalid-key, value-too-large, and operation-failed variants so components can handle bounded storage failures without relying on host-specific errors.
2026-07-29 19:37:00 +08:00
26 changed files with 1971 additions and 56 deletions
Generated
+7
View File
@@ -1995,6 +1995,13 @@ dependencies = [
"wasm-bindgen", "wasm-bindgen",
] ]
[[package]]
name = "kv-probe-component"
version = "0.1.0"
dependencies = [
"wit-bindgen 0.41.0",
]
[[package]] [[package]]
name = "lazy_static" name = "lazy_static"
version = "1.5.0" version = "1.5.0"
+1
View File
@@ -8,6 +8,7 @@ members = [
"components/fault", "components/fault",
"components/spin", "components/spin",
"components/clock-probe", "components/clock-probe",
"components/kv-probe",
"components/wasi-clock-probe", "components/wasi-clock-probe",
] ]
default-members = ["crates/wasmeld-runtime"] default-members = ["crates/wasmeld-runtime"]
+37
View File
@@ -8,6 +8,11 @@ Runtime 内置版本化 Host Capability Registry。注册 Component 时会直接
imports,只链接实际请求且版本完全匹配的 Host 能力;未知能力会被拒绝。能力不需要在 imports,只链接实际请求且版本完全匹配的 Host 能力;未知能力会被拒绝。能力不需要在
`.wasmpkg` 中重复声明,管理面会展示每个服务版本解析出的完整接口标识。 `.wasmpkg` 中重复声明,管理面会展示每个服务版本解析出的完整接口标识。
当前 Host 能力:
- `wasmeld:clock/monotonic-clock@0.1.0`Actor 内单调时钟
- `wasmeld:kv/store@0.1.0`:按服务隔离、跨 Revision 共享的持久化二进制 KV
## 结构 ## 结构
```text ```text
@@ -51,6 +56,25 @@ cd console
npm run dev npm run dev
``` ```
## 组件开发
Console 启动后,使用开发模式监听组件源码和本地 WIT `replace`
```bash
cargo run -p wasmeld-package --bin wasmeld -- \
dev components/echo/Cargo.toml
```
首次构建和每次源码变化都会自动完成 WIT 同步、Debug 编译、打包、注册和 Deployment
切换。默认使用 `echo-dev` 这类独立服务 ID,并根据 Component 内容生成
`0.1.0-dev.h<hash>` Revision,不会覆盖正式服务。新版本构建、校验或启动失败时,上一
版本继续运行;切换成功后旧开发 Revision 会被注销并释放。
管理面每 5 秒刷新一次,可以直接在调用面板预览新版本。只构建并部署一次可使用
`--once`;其它选项包括 `--id``--console``--release``--poll-ms`。管理 API
地址也可通过 `WASMELD_CONSOLE` 设置。默认允许本地 `replace` 更新 `wit.lock`;若开发
期间不允许 WIT 契约发生变化,再添加 `--locked`
构建组件: 构建组件:
```bash ```bash
@@ -58,6 +82,16 @@ cargo run -p wasmeld-package --bin wasmeld -- \
pack components/echo/Cargo.toml --locked pack components/echo/Cargo.toml --locked
``` ```
KV 示例组件:
```bash
cargo run -p wasmeld-package --bin wasmeld -- \
pack components/kv-probe/Cargo.toml --locked
```
`kv-probe` 接受 `set:<key>:<value>``get:<key>``delete:<key>`,用于验证 Host KV
能力;它不是公开 Gateway 的业务协议。
发布 WIT Package 发布 WIT Package
```bash ```bash
@@ -68,3 +102,6 @@ cargo run -p wasmeld-package --bin wasmeld -- \
组件在 `wasmeld.toml` 中声明 Registry 和精确版本依赖,`wasmeld wit fetch` 会递归 组件在 `wasmeld.toml` 中声明 Registry 和精确版本依赖,`wasmeld wit fetch` 会递归
解析依赖、生成 `wit/deps`,并将完整解析结果记录在 `wit.lock`。本地开发可使用 解析依赖、生成 `wit/deps`,并将完整解析结果记录在 `wit.lock`。本地开发可使用
`[replace]` 临时覆盖 Registry 来源。 `[replace]` 临时覆盖 Registry 来源。
完整命令、开发选项、组件包格式和 WIT 依赖说明见
[`wasmeld-package` CLI 文档](crates/wasmeld-package/README.md)。
+16
View File
@@ -0,0 +1,16 @@
[package]
name = "kv-probe-component"
version = "0.1.0"
edition.workspace = true
rust-version.workspace = true
license.workspace = true
[package.metadata.wasmeld]
id = "kv-probe"
world = "component:kv-probe/kv-probe-component@0.1.0"
[lib]
crate-type = ["cdylib"]
[dependencies]
wit-bindgen.workspace = true
+63
View File
@@ -0,0 +1,63 @@
mod bindings {
wit_bindgen::generate!({
path: "wit",
world: "kv-probe-component",
generate_all,
});
}
use bindings::wasmeld::kv::store;
struct KvProbe;
impl bindings::Guest for KvProbe {
fn init(_config: Vec<u8>) -> Result<(), bindings::ServiceError> {
Ok(())
}
fn invoke(input: Vec<u8>) -> Result<Vec<u8>, bindings::ServiceError> {
if let Some(key) = input.strip_prefix(b"get:") {
let key = parse_key(key)?;
return store::get(&key)
.map(|value| value.unwrap_or_default())
.map_err(kv_error);
}
if let Some(command) = input.strip_prefix(b"set:") {
let separator = command
.iter()
.position(|byte| *byte == b':')
.ok_or_else(|| {
bindings::ServiceError::CallFailed(
"set command must be set:<key>:<value>".to_owned(),
)
})?;
let key = parse_key(&command[..separator])?;
let value = command[separator + 1..].to_vec();
store::set(&key, &value).map_err(kv_error)?;
return Ok(Vec::new());
}
if let Some(key) = input.strip_prefix(b"delete:") {
let key = parse_key(key)?;
store::delete(&key).map_err(kv_error)?;
return Ok(Vec::new());
}
Err(bindings::ServiceError::CallFailed(
"expected get:<key>, set:<key>:<value>, or delete:<key>".to_owned(),
))
}
}
fn parse_key(bytes: &[u8]) -> Result<String, bindings::ServiceError> {
core::str::from_utf8(bytes)
.map(str::to_owned)
.map_err(|_| bindings::ServiceError::CallFailed("key must be valid UTF-8".to_owned()))
}
fn kv_error(error: store::KvError) -> bindings::ServiceError {
bindings::ServiceError::CallFailed(format!("KV operation failed: {error:?}"))
}
bindings::export!(KvProbe with_types_in bindings);
+11
View File
@@ -0,0 +1,11 @@
schema_version = 1
[dependencies]
"wasmeld:kv" = "0.1.0"
"wasmeld:service" = "0.1.0"
[replace."wasmeld:kv"]
path = "../../wit/kv"
[replace."wasmeld:service"]
path = "../../wit/service"
+15
View File
@@ -0,0 +1,15 @@
schema_version = 1
[[package]]
name = "wasmeld:kv"
version = "0.1.0"
source = "path+../../wit/kv"
sha256 = "6dc3a84dc03c7104aa8b305518466c95e4440e1031c4bef3b59843f40f24d0fb"
replaced = true
[[package]]
name = "wasmeld:service"
version = "0.1.0"
source = "path+../../wit/service"
sha256 = "d5497307bbcd1e159f7707f385b488a6f2e26362d5256d5bc1080ac603a51305"
replaced = true
+6
View File
@@ -0,0 +1,6 @@
package component:kv-probe@0.1.0;
world kv-probe-component {
include wasmeld:service/service-component@0.1.0;
import wasmeld:kv/store@0.1.0;
}
+1 -1
View File
@@ -35,7 +35,7 @@ export type BackendService = {
export type BackendEvent = { export type BackendEvent = {
id: number; id: number;
timestamp_ms: number; timestamp_ms: number;
kind: "registered" | "deployed" | "started" | "stopped" | "invoked" | "failed"; kind: "registered" | "unregistered" | "deployed" | "started" | "stopped" | "invoked" | "failed";
service_id: string | null; service_id: string | null;
revision: string | null; revision: string | null;
message: string; message: string;
+1
View File
@@ -135,6 +135,7 @@ function toEvent(event: BackendEvent): RuntimeEvent {
event.service_id && event.revision ? `${event.service_id}@${event.revision}` : "Wasmeld"; event.service_id && event.revision ? `${event.service_id}@${event.revision}` : "Wasmeld";
const meta: Record<BackendEvent["kind"], { label: string; tone: EventTone }> = { const meta: Record<BackendEvent["kind"], { label: string; tone: EventTone }> = {
registered: { label: "已注册", tone: "success" }, registered: { label: "已注册", tone: "success" },
unregistered: { label: "已注销", tone: "neutral" },
deployed: { label: "部署切换", tone: "success" }, deployed: { label: "部署切换", tone: "success" },
started: { label: "已启动", tone: "success" }, started: { label: "已启动", tone: "success" },
stopped: { label: "已停止", tone: "neutral" }, stopped: { label: "已停止", tone: "neutral" },
+15 -4
View File
@@ -8,11 +8,14 @@
- 启动、停止和重启常驻 Actor - 启动、停止和重启常驻 Actor
- 管理服务 ID 到活动版本的 Deployment - 管理服务 ID 到活动版本的 Deployment
- 通过独立 Gateway 转发原始二进制调用 - 通过独立 Gateway 转发原始二进制调用
- 为导入 `wasmeld:kv/store@0.1.0` 的 Component 提供持久化 KV
- 暴露调用计数、状态和最近事件 - 暴露调用计数、状态和最近事件
Wasm 制品保存在本地文件系统。服务 manifest、调用计数和最近 256 条事件通过 Wasm 制品保存在本地文件系统。服务 manifest、调用计数和最近 256 条事件通过
[Toasty](https://github.com/tokio-rs/toasty) 写入本地 libSQL 数据库,默认路径是 [Toasty](https://github.com/tokio-rs/toasty) 写入本地 libSQL 数据库,默认路径是
`var/wasmeld/console.db`。Deployment 也存储在同一数据库中。 `var/wasmeld/console.db`。Deployment 也存储在同一数据库中。
Host KV 使用 `(service_id, key)` 复合主键存储在该数据库中:同一服务的 Revision
共享数据,不同服务互相隔离,Runtime 或 Console 重启不会清空 KV。
`wasmeld-console` 启动时会在同一进程内创建 `wasmeld-runtime`、加载 Wasmtime Engine `wasmeld-console` 启动时会在同一进程内创建 `wasmeld-runtime`、加载 Wasmtime Engine
并重新注册已保存的 Component。Runtime 停止后 Console HTTP 和数据库仍然在线; 并重新注册已保存的 Component。Runtime 停止后 Console HTTP 和数据库仍然在线;
@@ -57,6 +60,7 @@ WIT Registry 路由。
| `POST` | `/api/v1/runtime/restart` | 重建 Runtime,并重新创建活动 Deployment 的 Actor | | `POST` | `/api/v1/runtime/restart` | 重建 Runtime,并重新创建活动 Deployment 的 Actor |
| `GET` | `/api/v1/services` | 服务版本及精确 Host Capability 列表 | | `GET` | `/api/v1/services` | 服务版本及精确 Host Capability 列表 |
| `POST` | `/api/v1/services` | multipart 上传单个 `package` 字段(`.wasmpkg` | | `POST` | `/api/v1/services` | multipart 上传单个 `package` 字段(`.wasmpkg` |
| `DELETE` | `/api/v1/services/{id}/{revision}` | 注销非活动 Revision 并删除制品 |
| `POST` | `/api/v1/services/{id}/{revision}/start` | 启动 Actor | | `POST` | `/api/v1/services/{id}/{revision}/start` | 启动 Actor |
| `POST` | `/api/v1/services/{id}/{revision}/stop` | 停止 Actor | | `POST` | `/api/v1/services/{id}/{revision}/stop` | 停止 Actor |
| `POST` | `/api/v1/services/{id}/{revision}/restart` | 重启 Actor | | `POST` | `/api/v1/services/{id}/{revision}/restart` | 重启 Actor |
@@ -70,6 +74,9 @@ WIT Registry 路由。
| `GET` | `/api/v1/wit/packages/{namespace}/{name}/{version}` | WIT 包元数据 | | `GET` | `/api/v1/wit/packages/{namespace}/{name}/{version}` | WIT 包元数据 |
| `GET` | `/api/v1/wit/packages/{namespace}/{name}/{version}/content` | 下载 WIT 包 | | `GET` | `/api/v1/wit/packages/{namespace}/{name}/{version}/content` | 下载 WIT 包 |
活动 Deployment 不能直接注销。开发工具会先完成新 Revision 的编译、注册和切换,
再注销上一份开发 Revision,因此失败的构建不会影响当前可调用版本。
激活已注册版本: 激活已注册版本:
```bash ```bash
@@ -95,7 +102,11 @@ curl http://127.0.0.1:8081/v1/services/echo/invoke \
`wasmeld:clock/monotonic-clock@0.1.0`。Console 不接受手工能力声明,Runtime 只会链接 `wasmeld:clock/monotonic-clock@0.1.0`。Console 不接受手工能力声明,Runtime 只会链接
注册表中存在且版本完全匹配的接口。 注册表中存在且版本完全匹配的接口。
组件包的构建与格式说明见 ## Host KV
[`docs/design/wasmeld-component-package.md`](../../docs/design/wasmeld-component-package.md)。
WIT 依赖、Registry 和本地 replace 说明见 `wasmeld:kv/store@0.1.0` 提供 `get``set``delete`。Key 必须是非空 UTF-8
[`docs/design/wit-package-management.md`](../../docs/design/wit-package-management.md)。 最大 256 BValue 最大 64 KiB。数据库命令通过有界专用工作线程执行,Host 等待时间
不会超过服务的 `deadline_ms`。v0.1.0 不提供 TTL、遍历、事务或 CAS。
组件包格式、开发循环、WIT 依赖、Registry 和本地 `replace` 说明见
[`wasmeld-package` CLI 文档](../wasmeld-package/README.md)。
+219
View File
@@ -0,0 +1,219 @@
//! Bounded bridge between synchronous Component Host calls and async Toasty.
use std::{
fmt, io,
sync::{
Arc, Mutex,
mpsc::{self, Receiver, SyncSender, TrySendError},
},
thread,
time::Duration,
};
use wasmeld_runtime::{KvBackend, KvBackendError};
use crate::persistence::Persistence;
const COMMAND_CAPACITY: usize = 64;
const MAX_RESPONSE_TIMEOUT: Duration = Duration::from_millis(400);
pub(crate) struct ToastyKvBackend {
inner: Arc<BackendInner>,
}
struct BackendInner {
sender: SyncSender<KvCommand>,
worker: Mutex<Option<thread::JoinHandle<()>>>,
}
enum KvCommand {
Get {
service_id: String,
key: String,
response: SyncSender<Result<Option<Vec<u8>>, KvBackendError>>,
},
Set {
service_id: String,
key: String,
value: Vec<u8>,
response: SyncSender<Result<(), KvBackendError>>,
},
Delete {
service_id: String,
key: String,
response: SyncSender<Result<(), KvBackendError>>,
},
Shutdown,
}
impl ToastyKvBackend {
pub(crate) fn start(persistence: Persistence) -> io::Result<Self> {
let (sender, receiver) = mpsc::sync_channel(COMMAND_CAPACITY);
let (ready_sender, ready_receiver) = mpsc::sync_channel(1);
let worker = thread::Builder::new()
.name("wasmeld-kv-backend".to_owned())
.spawn(move || run_worker(persistence, receiver, ready_sender))?;
match ready_receiver.recv() {
Ok(Ok(())) => Ok(Self {
inner: Arc::new(BackendInner {
sender,
worker: Mutex::new(Some(worker)),
}),
}),
Ok(Err(error)) => {
let _ = worker.join();
Err(error)
}
Err(_) => {
let _ = worker.join();
Err(io::Error::other("KV backend worker stopped during startup"))
}
}
}
fn request<T>(
&self,
timeout: Duration,
build: impl FnOnce(SyncSender<Result<T, KvBackendError>>) -> KvCommand,
) -> Result<T, KvBackendError> {
let (response, receiver) = mpsc::sync_channel(1);
match self.inner.sender.try_send(build(response)) {
Ok(()) => {}
Err(TrySendError::Full(_)) => return Err(KvBackendError::Busy),
Err(TrySendError::Disconnected(_)) => {
return Err(KvBackendError::Operation(
"backend worker is unavailable".to_owned(),
));
}
}
receiver
.recv_timeout(timeout.min(MAX_RESPONSE_TIMEOUT))
.map_err(|_| KvBackendError::Timeout)?
}
}
impl Clone for ToastyKvBackend {
fn clone(&self) -> Self {
Self {
inner: Arc::clone(&self.inner),
}
}
}
impl fmt::Debug for ToastyKvBackend {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
formatter
.debug_struct("ToastyKvBackend")
.finish_non_exhaustive()
}
}
impl KvBackend for ToastyKvBackend {
fn get(
&self,
service_id: &str,
key: &str,
timeout: Duration,
) -> Result<Option<Vec<u8>>, KvBackendError> {
self.request(timeout, |response| KvCommand::Get {
service_id: service_id.to_owned(),
key: key.to_owned(),
response,
})
}
fn set(
&self,
service_id: &str,
key: &str,
value: &[u8],
timeout: Duration,
) -> Result<(), KvBackendError> {
self.request(timeout, |response| KvCommand::Set {
service_id: service_id.to_owned(),
key: key.to_owned(),
value: value.to_vec(),
response,
})
}
fn delete(&self, service_id: &str, key: &str, timeout: Duration) -> Result<(), KvBackendError> {
self.request(timeout, |response| KvCommand::Delete {
service_id: service_id.to_owned(),
key: key.to_owned(),
response,
})
}
}
impl Drop for BackendInner {
fn drop(&mut self) {
let _ = self.sender.send(KvCommand::Shutdown);
if let Ok(worker) = self.worker.get_mut()
&& let Some(worker) = worker.take()
{
let _ = worker.join();
}
}
}
fn run_worker(
persistence: Persistence,
receiver: Receiver<KvCommand>,
ready: SyncSender<io::Result<()>>,
) {
let runtime = match tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()
{
Ok(runtime) => runtime,
Err(error) => {
let _ = ready.send(Err(error));
return;
}
};
let _ = ready.send(Ok(()));
while let Ok(command) = receiver.recv() {
match command {
KvCommand::Get {
service_id,
key,
response,
} => {
let result = runtime
.block_on(persistence.kv_get(&service_id, &key))
.map_err(operation_failed);
let _ = response.send(result);
}
KvCommand::Set {
service_id,
key,
value,
response,
} => {
let result = runtime
.block_on(persistence.kv_set(service_id, key, value))
.map_err(operation_failed);
let _ = response.send(result);
}
KvCommand::Delete {
service_id,
key,
response,
} => {
let result = runtime
.block_on(persistence.kv_delete(&service_id, &key))
.map_err(operation_failed);
let _ = response.send(result);
}
KvCommand::Shutdown => return,
}
}
}
fn operation_failed(error: toasty::Error) -> KvBackendError {
tracing::error!(%error, "persistent KV operation failed");
KvBackendError::Operation("persistent storage failure".to_owned())
}
+84 -4
View File
@@ -6,6 +6,7 @@
//! package Registry. Actor memory remains process-local and is never presented //! package Registry. Actor memory remains process-local and is never presented
//! as durable state. //! as durable state.
mod kv_backend;
mod persistence; mod persistence;
mod wit_registry; mod wit_registry;
@@ -23,7 +24,7 @@ use axum::{
extract::{DefaultBodyLimit, Multipart, Path as AxumPath, State, rejection::BytesRejection}, extract::{DefaultBodyLimit, Multipart, Path as AxumPath, State, rejection::BytesRejection},
http::{HeaderMap, HeaderValue, Method, StatusCode, header}, http::{HeaderMap, HeaderValue, Method, StatusCode, header},
response::{IntoResponse, Response}, response::{IntoResponse, Response},
routing::{get, post}, routing::{delete, get, post},
}; };
use base64::{Engine as _, engine::general_purpose::STANDARD as BASE64}; use base64::{Engine as _, engine::general_purpose::STANDARD as BASE64};
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
@@ -43,6 +44,7 @@ use wasmeld_runtime::{
ServiceManifest, ServiceManifest,
}; };
use kv_backend::ToastyKvBackend;
use persistence::{Persistence, StoredDeployment, StoredEvent, StoredService}; use persistence::{Persistence, StoredDeployment, StoredEvent, StoredService};
use wit_registry::WitRegistry; use wit_registry::WitRegistry;
pub use wit_registry::WitRegistryError; pub use wit_registry::WitRegistryError;
@@ -216,6 +218,7 @@ pub struct EventView {
#[serde(rename_all = "snake_case")] #[serde(rename_all = "snake_case")]
pub enum EventKind { pub enum EventKind {
Registered, Registered,
Unregistered,
Deployed, Deployed,
Started, Started,
Stopped, Stopped,
@@ -227,6 +230,7 @@ impl EventKind {
fn as_database_value(self) -> &'static str { fn as_database_value(self) -> &'static str {
match self { match self {
Self::Registered => "registered", Self::Registered => "registered",
Self::Unregistered => "unregistered",
Self::Deployed => "deployed", Self::Deployed => "deployed",
Self::Started => "started", Self::Started => "started",
Self::Stopped => "stopped", Self::Stopped => "stopped",
@@ -238,6 +242,7 @@ impl EventKind {
fn from_database_value(value: &str) -> Result<Self, ConsoleError> { fn from_database_value(value: &str) -> Result<Self, ConsoleError> {
match value { match value {
"registered" => Ok(Self::Registered), "registered" => Ok(Self::Registered),
"unregistered" => Ok(Self::Unregistered),
"deployed" => Ok(Self::Deployed), "deployed" => Ok(Self::Deployed),
"started" => Ok(Self::Started), "started" => Ok(Self::Started),
"stopped" => Ok(Self::Stopped), "stopped" => Ok(Self::Stopped),
@@ -349,6 +354,9 @@ pub enum ConsoleError {
#[error("service {0} has no active deployment")] #[error("service {0} has no active deployment")]
DeploymentNotFound(String), DeploymentNotFound(String),
#[error("service revision {0} is the active deployment")]
ActiveDeployment(ServiceKey),
#[error("Wasmeld Runtime is not running")] #[error("Wasmeld Runtime is not running")]
RuntimeNotRunning, RuntimeNotRunning,
@@ -374,6 +382,9 @@ pub enum ConsoleError {
#[error("database operation failed: {0}")] #[error("database operation failed: {0}")]
Database(#[from] toasty::Error), Database(#[from] toasty::Error),
#[error("failed to start persistent KV backend: {0}")]
KvBackendStart(#[source] io::Error),
#[error("database contains invalid console data: {0}")] #[error("database contains invalid console data: {0}")]
InvalidDatabaseData(String), InvalidDatabaseData(String),
@@ -410,6 +421,13 @@ impl Console {
})?; })?;
} }
let persistence = Persistence::open(&config.database_path).await?; let persistence = Persistence::open(&config.database_path).await?;
let mut runtime_config = config.runtime;
if runtime_config.kv_backend.is_none() {
runtime_config.kv_backend = Some(Arc::new(
ToastyKvBackend::start(persistence.clone())
.map_err(ConsoleError::KvBackendStart)?,
));
}
let wit_registry = let wit_registry =
WitRegistry::open(&config.wit_registry_dir, config.max_wit_package_bytes)?; WitRegistry::open(&config.wit_registry_dir, config.max_wit_package_bytes)?;
let (stored_services, mut stored_events, stored_deployments) = persistence.load().await?; let (stored_services, mut stored_events, stored_deployments) = persistence.load().await?;
@@ -424,10 +442,10 @@ impl Console {
.map(EventView::try_from) .map(EventView::try_from)
.collect::<Result<VecDeque<_>, _>>()?; .collect::<Result<VecDeque<_>, _>>()?;
let runtime = Runtime::new(config.runtime.clone())?; let runtime = Runtime::new(runtime_config.clone())?;
let console = Self { let console = Self {
runtime: RwLock::new(Some(runtime)), runtime: RwLock::new(Some(runtime)),
runtime_config: config.runtime, runtime_config,
artifact_dir, artifact_dir,
wit_registry, wit_registry,
max_artifact_bytes: config.max_artifact_bytes, max_artifact_bytes: config.max_artifact_bytes,
@@ -778,6 +796,50 @@ impl Console {
Ok(view) Ok(view)
} }
fn unregister(&self, key: &ServiceKey) -> Result<ServiceView, ConsoleError> {
let _lifecycle = self
.lifecycle
.lock()
.map_err(|_| ConsoleError::LockPoisoned)?;
let view = {
let state = self.state()?;
if state
.deployments
.get(key.id())
.is_some_and(|deployment| deployment.active_revision == key.revision())
{
return Err(ConsoleError::ActiveDeployment(key.clone()));
}
state
.services
.get(key)
.map(ServiceRecord::view)
.ok_or_else(|| ConsoleError::ServiceNotFound(key.clone()))?
};
let runtime = self.runtime()?;
let runtime = runtime.as_ref().ok_or(ConsoleError::RuntimeNotRunning)?;
runtime.unregister(key)?;
self.state()?.services.remove(key);
for path in [
self.artifact_dir.join(format!("{key}.wasm")),
self.artifact_dir.join(format!("{key}.toml")),
] {
if let Err(error) = fs::remove_file(&path)
&& error.kind() != io::ErrorKind::NotFound
{
tracing::warn!(path = %path.display(), %error, "failed to remove unregistered artifact");
}
}
self.push_event(
EventKind::Unregistered,
Some(key),
"component revision unregistered".to_owned(),
)?;
Ok(view)
}
fn start(&self, key: &ServiceKey, init_config: Vec<u8>) -> Result<ServiceView, ConsoleError> { fn start(&self, key: &ServiceKey, init_config: Vec<u8>) -> Result<ServiceView, ConsoleError> {
let runtime = self.runtime()?; let runtime = self.runtime()?;
let runtime = runtime.as_ref().ok_or(ConsoleError::RuntimeNotRunning)?; let runtime = runtime.as_ref().ok_or(ConsoleError::RuntimeNotRunning)?;
@@ -1209,7 +1271,7 @@ pub fn app(console: Arc<Console>, allowed_origins: Vec<HeaderValue>) -> Router {
.max(console.max_wit_package_bytes()) .max(console.max_wit_package_bytes())
.saturating_add(1024 * 1024); .saturating_add(1024 * 1024);
let mut cors = CorsLayer::new() let mut cors = CorsLayer::new()
.allow_methods([Method::GET, Method::POST]) .allow_methods([Method::GET, Method::POST, Method::DELETE])
.allow_headers([header::CONTENT_TYPE]); .allow_headers([header::CONTENT_TYPE]);
if !allowed_origins.is_empty() { if !allowed_origins.is_empty() {
cors = cors.allow_origin(allowed_origins); cors = cors.allow_origin(allowed_origins);
@@ -1222,6 +1284,10 @@ pub fn app(console: Arc<Console>, allowed_origins: Vec<HeaderValue>) -> Router {
.route("/api/v1/runtime/stop", post(stop_runtime)) .route("/api/v1/runtime/stop", post(stop_runtime))
.route("/api/v1/runtime/restart", post(restart_runtime)) .route("/api/v1/runtime/restart", post(restart_runtime))
.route("/api/v1/services", get(list_services).post(register)) .route("/api/v1/services", get(list_services).post(register))
.route(
"/api/v1/services/{id}/{revision}",
delete(unregister_service),
)
.route("/api/v1/deployments", get(list_deployments)) .route("/api/v1/deployments", get(list_deployments))
.route("/api/v1/deployments/{id}", get(get_deployment)) .route("/api/v1/deployments/{id}", get(get_deployment))
.route( .route(
@@ -1561,6 +1627,16 @@ async fn stop_service(
)) ))
} }
async fn unregister_service(
State(console): State<Arc<Console>>,
AxumPath((id, revision)): AxumPath<(String, String)>,
) -> Result<Json<ServiceView>, ApiError> {
let key = api_key(id, revision)?;
Ok(Json(
run_blocking(console, move |console| console.unregister(&key)).await?,
))
}
async fn restart_service( async fn restart_service(
State(console): State<Arc<Console>>, State(console): State<Arc<Console>>,
AxumPath((id, revision)): AxumPath<(String, String)>, AxumPath((id, revision)): AxumPath<(String, String)>,
@@ -1754,11 +1830,13 @@ impl IntoResponse for GatewayError {
| RuntimeError::ComponentError { .. }, | RuntimeError::ComponentError { .. },
) => (StatusCode::BAD_GATEWAY, "service_error", error.to_string()), ) => (StatusCode::BAD_GATEWAY, "service_error", error.to_string()),
ConsoleError::ArtifactTooLarge { .. } ConsoleError::ArtifactTooLarge { .. }
| ConsoleError::ActiveDeployment(_)
| ConsoleError::Package(_) | ConsoleError::Package(_)
| ConsoleError::WitRegistry(_) | ConsoleError::WitRegistry(_)
| ConsoleError::Storage { .. } | ConsoleError::Storage { .. }
| ConsoleError::ManifestSerialize(_) | ConsoleError::ManifestSerialize(_)
| ConsoleError::Database(_) | ConsoleError::Database(_)
| ConsoleError::KvBackendStart(_)
| ConsoleError::InvalidDatabaseData(_) | ConsoleError::InvalidDatabaseData(_)
| ConsoleError::Runtime(_) | ConsoleError::Runtime(_)
| ConsoleError::Join(_) | ConsoleError::Join(_)
@@ -1805,6 +1883,7 @@ impl IntoResponse for ApiError {
ConsoleError::InvalidRequest(_) => (StatusCode::BAD_REQUEST, "invalid_request"), ConsoleError::InvalidRequest(_) => (StatusCode::BAD_REQUEST, "invalid_request"),
ConsoleError::ServiceNotFound(_) => (StatusCode::NOT_FOUND, "service_not_found"), ConsoleError::ServiceNotFound(_) => (StatusCode::NOT_FOUND, "service_not_found"),
ConsoleError::DeploymentNotFound(_) => (StatusCode::NOT_FOUND, "deployment_not_found"), ConsoleError::DeploymentNotFound(_) => (StatusCode::NOT_FOUND, "deployment_not_found"),
ConsoleError::ActiveDeployment(_) => (StatusCode::CONFLICT, "active_deployment"),
ConsoleError::RuntimeNotRunning => (StatusCode::CONFLICT, "runtime_not_running"), ConsoleError::RuntimeNotRunning => (StatusCode::CONFLICT, "runtime_not_running"),
ConsoleError::ArtifactTooLarge { .. } ConsoleError::ArtifactTooLarge { .. }
| ConsoleError::Package(PackageError::ComponentTooLarge { .. }) | ConsoleError::Package(PackageError::ComponentTooLarge { .. })
@@ -1849,6 +1928,7 @@ impl IntoResponse for ApiError {
ConsoleError::Storage { .. } ConsoleError::Storage { .. }
| ConsoleError::ManifestSerialize(_) | ConsoleError::ManifestSerialize(_)
| ConsoleError::Database(_) | ConsoleError::Database(_)
| ConsoleError::KvBackendStart(_)
| ConsoleError::InvalidDatabaseData(_) | ConsoleError::InvalidDatabaseData(_)
| ConsoleError::Runtime(_) | ConsoleError::Runtime(_)
| ConsoleError::WitRegistry(WitRegistryError::InvalidStorage(_)) | ConsoleError::WitRegistry(WitRegistryError::InvalidStorage(_))
+98 -7
View File
@@ -1,10 +1,15 @@
//! Toasty models and transactional libSQL snapshots for Console control data. //! Toasty models and transactional libSQL snapshots for Console control data.
//! //!
//! Component bytes and WIT packages remain filesystem artifacts. This module //! Component bytes and WIT packages remain filesystem artifacts. This module
//! persists only service manifests, deployments, metrics, and the bounded //! persists service manifests, deployments, metrics, the bounded event
//! event history. //! history, and service-scoped Host KV entries.
use std::path::Path; use std::{
collections::BTreeSet,
path::Path,
sync::Arc,
time::{SystemTime, UNIX_EPOCH},
};
use toasty::Db; use toasty::Db;
use toasty_driver_turso::Turso; use toasty_driver_turso::Turso;
@@ -40,9 +45,19 @@ pub(crate) struct StoredDeployment {
pub updated_at_ms: u64, pub updated_at_ms: u64,
} }
#[derive(Clone, Debug, toasty::Model)]
#[key(service_id, entry_key)]
pub(crate) struct StoredKvEntry {
pub service_id: String,
pub entry_key: String,
pub value: Vec<u8>,
pub updated_at_ms: u64,
}
/// Serialized access to the Toasty database connection. /// Serialized access to the Toasty database connection.
#[derive(Clone)]
pub(crate) struct Persistence { pub(crate) struct Persistence {
db: Mutex<Db>, db: Arc<Mutex<Db>>,
} }
impl Persistence { impl Persistence {
@@ -53,7 +68,8 @@ impl Persistence {
.models(toasty::models!( .models(toasty::models!(
StoredService, StoredService,
StoredEvent, StoredEvent,
StoredDeployment StoredDeployment,
StoredKvEntry
)) ))
.build(Turso::file(path)) .build(Turso::file(path))
.await?; .await?;
@@ -74,9 +90,24 @@ impl Persistence {
) )
.exec(&mut db) .exec(&mut db)
.await?; .await?;
return Ok(Self { db: Mutex::new(db) }); toasty::sql::statement(
r#"CREATE TABLE IF NOT EXISTS "stored_kv_entries" (
"service_id" TEXT NOT NULL,
"entry_key" TEXT NOT NULL,
"value" BLOB NOT NULL,
"updated_at_ms" INTEGER NOT NULL,
PRIMARY KEY ("service_id", "entry_key")
)"#,
)
.exec(&mut db)
.await?;
return Ok(Self {
db: Arc::new(Mutex::new(db)),
});
} }
Ok(Self { db: Mutex::new(db) }) Ok(Self {
db: Arc::new(Mutex::new(db)),
})
} }
/// Loads the complete service and deployment sets plus retained events. /// Loads the complete service and deployment sets plus retained events.
@@ -100,6 +131,15 @@ impl Persistence {
let mut db = self.db.lock().await; let mut db = self.db.lock().await;
let mut tx = db.transaction().await?; let mut tx = db.transaction().await?;
let service_keys = services
.iter()
.map(|service| service.service_key.clone())
.collect::<BTreeSet<_>>();
for stored in StoredService::all().exec(&mut tx).await? {
if !service_keys.contains(&stored.service_key) {
StoredService::delete_by_service_key(&mut tx, &stored.service_key).await?;
}
}
for service in services { for service in services {
StoredService::upsert_by_service_key(service.service_key) StoredService::upsert_by_service_key(service.service_key)
.manifest_toml(service.manifest_toml) .manifest_toml(service.manifest_toml)
@@ -129,6 +169,15 @@ impl Persistence {
.await?; .await?;
} }
let deployment_ids = deployments
.iter()
.map(|deployment| deployment.service_id.clone())
.collect::<BTreeSet<_>>();
for stored in StoredDeployment::all().exec(&mut tx).await? {
if !deployment_ids.contains(&stored.service_id) {
StoredDeployment::delete_by_service_id(&mut tx, &stored.service_id).await?;
}
}
for deployment in deployments { for deployment in deployments {
StoredDeployment::upsert_by_service_id(deployment.service_id) StoredDeployment::upsert_by_service_id(deployment.service_id)
.active_revision(deployment.active_revision) .active_revision(deployment.active_revision)
@@ -139,4 +188,46 @@ impl Persistence {
tx.commit().await tx.commit().await
} }
pub(crate) async fn kv_get(
&self,
service_id: &str,
entry_key: &str,
) -> toasty::Result<Option<Vec<u8>>> {
let mut db = self.db.lock().await;
Ok(
StoredKvEntry::filter_by_service_id_and_entry_key(service_id, entry_key)
.first()
.exec(&mut *db)
.await?
.map(|entry| entry.value),
)
}
pub(crate) async fn kv_set(
&self,
service_id: String,
entry_key: String,
value: Vec<u8>,
) -> toasty::Result<()> {
let mut db = self.db.lock().await;
StoredKvEntry::upsert_by_service_id_and_entry_key(service_id, entry_key)
.value(value)
.updated_at_ms(unix_time_ms())
.exec(&mut *db)
.await?;
Ok(())
}
pub(crate) async fn kv_delete(&self, service_id: &str, entry_key: &str) -> toasty::Result<()> {
let mut db = self.db.lock().await;
StoredKvEntry::delete_by_service_id_and_entry_key(&mut *db, service_id, entry_key).await
}
}
fn unix_time_ms() -> u64 {
SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|duration| u64::try_from(duration.as_millis()).unwrap_or(u64::MAX))
.unwrap_or_default()
} }
+178 -1
View File
@@ -261,6 +261,74 @@ async fn routes_raw_gateway_calls_through_the_active_deployment() {
assert_eq!(deployments["deployments"][0]["active_revision"], "0.2.0"); assert_eq!(deployments["deployments"][0]["active_revision"], "0.2.0");
} }
#[tokio::test]
async fn unregisters_only_inactive_service_revisions_and_prunes_persistence() {
let artifact_dir = TempDir::new().expect("temporary artifact directory");
let component = fs::read(echo_component()).expect("echo component should be readable");
{
let application = test_app(artifact_dir.path()).await;
for revision in ["0.1.0", "0.2.0"] {
let response = application
.clone()
.oneshot(package_request("dev-echo", revision, &component))
.await
.expect("register request should complete");
assert_eq!(response.status(), StatusCode::CREATED);
}
let response = application
.clone()
.oneshot(json_request(
"/api/v1/deployments/dev-echo/activate",
json!({ "revision": "0.1.0" }),
))
.await
.expect("activation request should complete");
assert_eq!(response.status(), StatusCode::OK);
let response = application
.clone()
.oneshot(delete_request("/api/v1/services/dev-echo/0.1.0"))
.await
.expect("active unregister request should complete");
assert_eq!(response.status(), StatusCode::CONFLICT);
assert_eq!(
response_json(response).await["error"]["code"],
"active_deployment"
);
let response = application
.clone()
.oneshot(json_request(
"/api/v1/deployments/dev-echo/activate",
json!({ "revision": "0.2.0" }),
))
.await
.expect("replacement activation should complete");
assert_eq!(response.status(), StatusCode::OK);
let response = application
.clone()
.oneshot(delete_request("/api/v1/services/dev-echo/0.1.0"))
.await
.expect("inactive unregister request should complete");
assert_eq!(response.status(), StatusCode::OK);
assert!(!artifact_dir.path().join("dev-echo@0.1.0.wasm").exists());
assert!(!artifact_dir.path().join("dev-echo@0.1.0.toml").exists());
}
let application = test_app(artifact_dir.path()).await;
let response = application
.oneshot(get_request("/api/v1/services"))
.await
.expect("restored service list request should complete");
let body = response_json(response).await;
let services = body["services"].as_array().unwrap();
assert_eq!(services.len(), 1);
assert_eq!(services[0]["revision"], "0.2.0");
}
#[tokio::test] #[tokio::test]
async fn restores_the_active_deployment_with_a_fresh_actor() { async fn restores_the_active_deployment_with_a_fresh_actor() {
let artifact_dir = TempDir::new().expect("temporary artifact directory"); let artifact_dir = TempDir::new().expect("temporary artifact directory");
@@ -369,6 +437,80 @@ async fn reloads_manifests_without_restoring_actor_memory() {
assert!(body["events"].as_array().unwrap().len() >= 3); assert!(body["events"].as_array().unwrap().len() >= 3);
} }
#[tokio::test]
async fn persists_service_scoped_kv_across_revisions_and_restarts() {
const KV_WORLD: &str = "component:kv-probe/kv-probe-component@0.1.0";
let artifact_dir = TempDir::new().expect("temporary artifact directory");
let component = fs::read(component_artifact("kv_probe_component.wasm"))
.expect("KV probe component should be readable");
{
let application = test_app(artifact_dir.path()).await;
for (id, revision) in [
("shared-kv", "0.1.0"),
("shared-kv", "0.2.0"),
("isolated-kv", "0.1.0"),
] {
let response = application
.clone()
.oneshot(package_request_with_world(
id, revision, KV_WORLD, &component,
))
.await
.expect("KV component registration should complete");
assert_eq!(response.status(), StatusCode::CREATED);
let registered = response_json(response).await;
assert_eq!(
registered["capabilities"][0]["interface"],
"wasmeld:kv/store@0.1.0"
);
let response = application
.clone()
.oneshot(empty_post(&format!(
"/api/v1/services/{id}/{revision}/start"
)))
.await
.expect("KV actor start should complete");
assert_eq!(response.status(), StatusCode::OK);
}
assert_eq!(
invoke_service(&application, "shared-kv", "0.1.0", b"set:answer:forty-two").await,
b""
);
assert_eq!(
invoke_service(&application, "shared-kv", "0.2.0", b"get:answer").await,
b"forty-two"
);
assert_eq!(
invoke_service(&application, "isolated-kv", "0.1.0", b"get:answer").await,
b""
);
}
let application = test_app(artifact_dir.path()).await;
let response = application
.clone()
.oneshot(empty_post("/api/v1/services/shared-kv/0.2.0/start"))
.await
.expect("restored KV actor start should complete");
assert_eq!(response.status(), StatusCode::OK);
assert_eq!(
invoke_service(&application, "shared-kv", "0.2.0", b"get:answer").await,
b"forty-two"
);
assert_eq!(
invoke_service(&application, "shared-kv", "0.2.0", b"delete:answer").await,
b""
);
assert_eq!(
invoke_service(&application, "shared-kv", "0.2.0", b"get:answer").await,
b""
);
}
#[tokio::test] #[tokio::test]
async fn manages_the_embedded_runtime_lifecycle() { async fn manages_the_embedded_runtime_lifecycle() {
let artifact_dir = TempDir::new().expect("temporary artifact directory"); let artifact_dir = TempDir::new().expect("temporary artifact directory");
@@ -719,7 +861,6 @@ async fn test_apps(artifact_dir: &Path) -> (Router, Router) {
} }
fn package_request(id: &str, revision: &str, component: &[u8]) -> Request<Body> { fn package_request(id: &str, revision: &str, component: &[u8]) -> Request<Body> {
let mut package = Cursor::new(Vec::new());
let world = if id == "counter" { let world = if id == "counter" {
"component:counter/counter-component@0.1.0" "component:counter/counter-component@0.1.0"
} else if id == "clock-probe" { } else if id == "clock-probe" {
@@ -727,6 +868,16 @@ fn package_request(id: &str, revision: &str, component: &[u8]) -> Request<Body>
} else { } else {
"component:echo/echo-component@0.1.0" "component:echo/echo-component@0.1.0"
}; };
package_request_with_world(id, revision, world, component)
}
fn package_request_with_world(
id: &str,
revision: &str,
world: &str,
component: &[u8],
) -> Request<Body> {
let mut package = Cursor::new(Vec::new());
write_package(&mut package, id, revision, world, component) write_package(&mut package, id, revision, world, component)
.expect("test package should be valid"); .expect("test package should be valid");
package_bytes_request(package.get_ref()) package_bytes_request(package.get_ref())
@@ -805,6 +956,14 @@ fn empty_post(uri: &str) -> Request<Body> {
.unwrap() .unwrap()
} }
fn delete_request(uri: &str) -> Request<Body> {
Request::builder()
.method("DELETE")
.uri(uri)
.body(Body::empty())
.unwrap()
}
fn get_request(uri: &str) -> Request<Body> { fn get_request(uri: &str) -> Request<Body> {
Request::builder().uri(uri).body(Body::empty()).unwrap() Request::builder().uri(uri).body(Body::empty()).unwrap()
} }
@@ -816,6 +975,23 @@ async fn response_json(response: axum::response::Response) -> Value {
serde_json::from_slice(&bytes).expect("response should be JSON") serde_json::from_slice(&bytes).expect("response should be JSON")
} }
async fn invoke_service(application: &Router, id: &str, revision: &str, input: &[u8]) -> Vec<u8> {
let response = application
.clone()
.oneshot(json_request(
&format!("/api/v1/services/{id}/{revision}/invoke"),
json!({ "input_base64": BASE64.encode(input) }),
))
.await
.expect("component invocation should complete");
let status = response.status();
let body = response_json(response).await;
assert_eq!(status, StatusCode::OK, "{body}");
BASE64
.decode(body["output_base64"].as_str().unwrap())
.unwrap()
}
fn echo_component() -> PathBuf { fn echo_component() -> PathBuf {
component_artifact("echo_component.wasm") component_artifact("echo_component.wasm")
} }
@@ -826,6 +1002,7 @@ fn component_artifact(name: &str) -> PathBuf {
"components/echo/Cargo.toml", "components/echo/Cargo.toml",
"components/counter/Cargo.toml", "components/counter/Cargo.toml",
"components/clock-probe/Cargo.toml", "components/clock-probe/Cargo.toml",
"components/kv-probe/Cargo.toml",
] { ] {
let status = Command::new("rustup") let status = Command::new("rustup")
.args([ .args([
+223
View File
@@ -0,0 +1,223 @@
# Wasmeld CLI
`wasmeld-package` 提供 `wasmeld` 命令,用于:
- 开发、编译和打包 WebAssembly Component
- 监听源码并将开发版本自动部署到 Wasmeld Console
- 解析、锁定和替换 WIT Package 依赖
- 构建并发布版本化 WIT Package
Runtime 不接收 Rust 源码。CLI 会先将 Rust 编译为 WebAssembly Component,再生成
`.wasmpkg` 或上传到 Console。
## 运行
在项目工作区内可以直接运行:
```bash
cargo +stable run -p wasmeld-package --bin wasmeld -- <command>
```
Component 默认使用 Rust `1.90.0``wasm32-wasip2` 目标构建。可以通过
`WASMELD_COMPONENT_TOOLCHAIN` 指定其它已安装工具链。
## Component 项目
一个可打包的 Component 至少需要:
```text
component/
├── Cargo.toml
├── wasmeld.toml
├── wit.lock
├── src/lib.rs
└── wit/world.wit
```
`Cargo.toml` 必须提供 `cdylib` Target 和 Wasmeld 元数据:
```toml
[package]
name = "echo-component"
version = "0.1.0"
[package.metadata.wasmeld]
id = "echo"
world = "component:echo/echo-component@0.1.0"
[lib]
crate-type = ["cdylib"]
```
`version` 成为正式包的 Revision`id` 是服务 ID`world` 必须与 Component 实际导出的
WIT World 一致。
## 边写边预览
先启动 Console
```bash
cargo +stable run -p wasmeld-console
```
再启动开发循环:
```bash
cargo +stable run -p wasmeld-package --bin wasmeld -- \
dev components/echo/Cargo.toml
```
开发循环会监听 Rust、Cargo、WIT 清单以及本地 `replace` WIT 源,并执行:
```text
同步 WIT 依赖
→ Debug 编译
→ 生成 .wasmpkg
→ 注册开发 Revision
→ 启动并切换 Deployment
→ 注销上一开发 Revision
```
默认服务 ID 为 `<id>-dev`Revision 为 `<version>-dev.h<component-hash>`。正式服务不会
被覆盖。构建、校验或启动失败时不会切换 Deployment,上一版本继续提供服务。
管理面默认每 5 秒刷新一次,也可以直接调用开发服务:
```bash
curl http://127.0.0.1:8081/v1/services/echo-dev/invoke \
-H 'Content-Type: application/octet-stream' \
--data-binary 'hello'
```
开发选项:
| 选项 | 作用 |
| --- | --- |
| `--console <url>` | Console 管理 API,默认 `http://127.0.0.1:8080` |
| `--id <service-id>` | 覆盖默认的 `<id>-dev` 服务 ID |
| `--once` | 构建、部署一次后退出,适合脚本和 CI |
| `--release` | 使用 Release 而不是 Debug 构建 |
| `--locked` | 要求依赖与现有 `wit.lock` 完全一致 |
| `--poll-ms <n>` | 源码扫描间隔,最小 100 ms,默认 350 ms |
也可以通过 `WASMELD_CONSOLE` 设置管理 API 地址。退出 `wasmeld dev` 不会停止当前开发
Deployment;它会继续运行,直到手工停止、切换或注销。
只修改 Rust 源码且不允许契约变化时使用 `--locked`。需要同步修改本地 WIT `replace`
时不要使用该选项,CLI 会更新 `wit.lock`
## 打包 Component
```bash
cargo +stable run -p wasmeld-package --bin wasmeld -- \
pack components/echo/Cargo.toml --locked
```
默认执行 Release 构建并输出:
```text
dist/echo-0.1.0.wasmpkg
```
可用选项:
| 选项 | 作用 |
| --- | --- |
| `--output <path>``-o <path>` | 指定输出文件 |
| `--no-build` | 使用现有 Release Component 制品 |
| `--locked` | 禁止 WIT 依赖解析结果改变 |
`.wasmpkg` 是受约束的 ZIP,固定包含:
```text
package.toml
component.wasm
```
`package.toml` 记录服务 ID、Revision、World 和 Component SHA-256。运行时资源限制不由
组件声明,而是在 Console 注册时由平台策略注入。
## WIT 依赖
`wasmeld.toml` 使用精确版本:
```toml
schema_version = 1
[wit]
root = "wit"
[registry]
url = "http://127.0.0.1:8080"
[dependencies]
"wasmeld:service" = "0.1.0"
"wasmeld:kv" = "0.1.0"
[replace."wasmeld:service"]
path = "../../wit/service"
```
依赖解析结果写入 `wit.lock`,源码开发时应提交该文件。解析出的 `wit/deps/` 是生成目录,
不应提交。
同步依赖:
```bash
wasmeld wit fetch --manifest components/echo/wasmeld.toml
```
校验锁定结果且禁止变化:
```bash
wasmeld wit fetch --manifest components/echo/wasmeld.toml --locked
```
`wit tidy` 当前与 `wit fetch` 使用相同的确定性解析和物化流程。查看根依赖和有效来源:
```bash
wasmeld wit graph --manifest components/echo/wasmeld.toml
```
添加本地替换:
```bash
wasmeld wit replace wasmeld:service@0.1.0 \
--path ../../wit/service \
--manifest components/echo/wasmeld.toml
```
包级替换可以省略 `@version`。精确版本替换的优先级更高。移除替换:
```bash
wasmeld wit replace wasmeld:service@0.1.0 \
--drop \
--manifest components/echo/wasmeld.toml
```
## WIT Package
将 WIT 源目录构建为标准二进制 WIT Package
```bash
wasmeld wit build wit/service \
--output dist/wit/wasmeld-service-0.1.0.wasm
```
发布到 Console 的不可变 WIT Registry
```bash
wasmeld wit publish wit/service \
--registry http://127.0.0.1:8080
```
Registry 地址也可以通过 `WASMELD_REGISTRY` 设置。相同 Package 版本不可覆盖;协议变化
应发布新版本,并由 Component 在 `wasmeld.toml` 中显式选择。
## 环境变量
| 变量 | 作用 |
| --- | --- |
| `WASMELD_COMPONENT_TOOLCHAIN` | Component Rust 工具链,默认 `1.90.0` |
| `WASMELD_CONSOLE` | `wasmeld dev` 使用的 Console 管理 API |
| `WASMELD_REGISTRY` | `wit publish` 使用的 Registry 地址 |
+431
View File
@@ -0,0 +1,431 @@
//! Local build, watch, and deployment loop for Component development.
use std::{
collections::BTreeSet,
env, fs, io,
path::{Path, PathBuf},
thread,
time::Duration,
};
use reqwest::{
StatusCode,
blocking::{Client, multipart},
};
use serde::Deserialize;
use sha2::{Digest, Sha256};
use wasmeld_package::module::{MODULE_MANIFEST_FILE, ModuleManifest};
use crate::{BuildProfile, PackRequest, PackageIdentity, PackedComponent, pack_component};
const DEFAULT_CONSOLE: &str = "http://127.0.0.1:8080";
const DEFAULT_POLL_MS: u64 = 350;
struct DevOptions {
manifest_path: PathBuf,
console: String,
service_id: Option<String>,
locked: bool,
release: bool,
once: bool,
poll_interval: Duration,
}
#[derive(Debug, Deserialize)]
struct ServiceList {
services: Vec<ServiceIdentity>,
}
#[derive(Debug, Deserialize)]
struct ServiceIdentity {
id: String,
revision: String,
}
#[derive(Debug, Deserialize)]
struct Deployment {
active_revision: String,
}
pub(crate) fn run(arguments: Vec<String>) -> Result<(), Box<dyn std::error::Error>> {
let options = parse_options(arguments)?;
let manifest_path = fs::canonicalize(&options.manifest_path)?;
let watch_roots = watch_roots(&manifest_path)?;
let client = Client::builder()
.user_agent(format!("wasmeld-dev/{}", env!("CARGO_PKG_VERSION")))
.timeout(Duration::from_secs(120))
.build()?;
ensure_console(&client, &options.console)?;
let temporary_dir = env::temp_dir().join(format!("wasmeld-dev-{}", std::process::id()));
fs::create_dir_all(&temporary_dir)?;
let output = temporary_dir.join("component.wasmpkg");
let mut observed = None;
println!("watching:");
for root in &watch_roots {
println!(" {}", root.display());
}
println!("console: {}", options.console);
loop {
let current = source_fingerprint(&watch_roots)?;
if observed.as_ref() == Some(&current) {
thread::sleep(options.poll_interval);
continue;
}
println!("change detected; building Component");
let result = build_and_deploy(&client, &options, &manifest_path, &output);
observed = Some(source_fingerprint(&watch_roots)?);
match result {
Ok(packed) => {
println!(
"ready: {}@{} ({})",
packed.manifest.id, packed.manifest.revision, packed.manifest.sha256
);
if options.once {
return Ok(());
}
}
Err(error) if options.once => return Err(error),
Err(error) => {
eprintln!("wasmeld dev: {error}");
eprintln!("waiting for the next source change; the previous deployment is intact");
}
}
}
}
fn parse_options(arguments: Vec<String>) -> Result<DevOptions, Box<dyn std::error::Error>> {
let mut arguments = arguments.into_iter();
let manifest_path = arguments.next().ok_or("dev requires a Cargo.toml path")?;
let mut console = env::var("WASMELD_CONSOLE").unwrap_or_else(|_| DEFAULT_CONSOLE.to_owned());
let mut service_id = None;
let mut locked = false;
let mut release = false;
let mut once = false;
let mut poll_ms = DEFAULT_POLL_MS;
while let Some(argument) = arguments.next() {
match argument.as_str() {
"--console" => {
console = arguments.next().ok_or("--console requires an HTTP URL")?;
}
"--id" => {
service_id = Some(arguments.next().ok_or("--id requires a service ID")?);
}
"--locked" => locked = true,
"--release" => release = true,
"--once" => once = true,
"--poll-ms" => {
poll_ms = arguments
.next()
.ok_or("--poll-ms requires a number")?
.parse::<u64>()?;
if poll_ms < 100 {
return Err("--poll-ms must be at least 100".into());
}
}
_ => return Err(format!("unknown dev argument {argument:?}").into()),
}
}
Ok(DevOptions {
manifest_path: PathBuf::from(manifest_path),
console: console.trim_end_matches('/').to_owned(),
service_id,
locked,
release,
once,
poll_interval: Duration::from_millis(poll_ms),
})
}
fn build_and_deploy(
client: &Client,
options: &DevOptions,
manifest_path: &Path,
output: &Path,
) -> Result<PackedComponent, Box<dyn std::error::Error>> {
let packed = pack_component(PackRequest {
manifest_path: manifest_path.to_owned(),
output: Some(output.to_owned()),
no_build: false,
locked: options.locked,
profile: if options.release {
BuildProfile::Release
} else {
BuildProfile::Debug
},
identity: PackageIdentity::Development {
id: options.service_id.clone(),
},
})?;
let previous = active_revision(client, &options.console, &packed.manifest.id)?;
register(client, &options.console, &packed)?;
activate(
client,
&options.console,
&packed.manifest.id,
&packed.manifest.revision,
)?;
if let Some(previous) = previous
&& previous != packed.manifest.revision
&& is_matching_dev_revision(&previous, &packed.manifest.revision)
&& let Err(error) = unregister(client, &options.console, &packed.manifest.id, &previous)
{
eprintln!(
"wasmeld dev: deployed successfully but could not unregister {}@{}: {error}",
packed.manifest.id, previous
);
}
Ok(packed)
}
fn ensure_console(client: &Client, console: &str) -> Result<(), Box<dyn std::error::Error>> {
let response = client.get(format!("{console}/healthz")).send()?;
if response.status().is_success() {
Ok(())
} else {
Err(http_error("Console health check", response))
}
}
fn active_revision(
client: &Client,
console: &str,
service_id: &str,
) -> Result<Option<String>, Box<dyn std::error::Error>> {
let response = client
.get(format!("{console}/api/v1/deployments/{service_id}"))
.send()?;
match response.status() {
status if status.is_success() => Ok(Some(response.json::<Deployment>()?.active_revision)),
StatusCode::NOT_FOUND => Ok(None),
_ => Err(http_error("deployment lookup", response)),
}
}
fn register(
client: &Client,
console: &str,
packed: &PackedComponent,
) -> Result<(), Box<dyn std::error::Error>> {
let package = fs::read(&packed.output)?;
let part = multipart::Part::bytes(package)
.file_name("component.wasmpkg")
.mime_str("application/vnd.wasmeld.package")?;
let response = client
.post(format!("{console}/api/v1/services"))
.multipart(multipart::Form::new().part("package", part))
.send()?;
if response.status().is_success() {
return Ok(());
}
if response.status() == StatusCode::CONFLICT
&& service_exists(
client,
console,
&packed.manifest.id,
&packed.manifest.revision,
)?
{
return Ok(());
}
Err(http_error("Component registration", response))
}
fn service_exists(
client: &Client,
console: &str,
service_id: &str,
revision: &str,
) -> Result<bool, Box<dyn std::error::Error>> {
let response = client.get(format!("{console}/api/v1/services")).send()?;
if !response.status().is_success() {
return Err(http_error("service lookup", response));
}
Ok(response
.json::<ServiceList>()?
.services
.iter()
.any(|service| service.id == service_id && service.revision == revision))
}
fn activate(
client: &Client,
console: &str,
service_id: &str,
revision: &str,
) -> Result<(), Box<dyn std::error::Error>> {
let response = client
.post(format!(
"{console}/api/v1/deployments/{service_id}/activate"
))
.json(&serde_json::json!({ "revision": revision }))
.send()?;
if response.status().is_success() {
Ok(())
} else {
Err(http_error("development deployment", response))
}
}
fn unregister(
client: &Client,
console: &str,
service_id: &str,
revision: &str,
) -> Result<(), Box<dyn std::error::Error>> {
let response = client
.delete(format!("{console}/api/v1/services/{service_id}/{revision}"))
.send()?;
if response.status().is_success() || response.status() == StatusCode::NOT_FOUND {
Ok(())
} else {
Err(http_error("old development revision cleanup", response))
}
}
fn http_error(action: &str, response: reqwest::blocking::Response) -> Box<dyn std::error::Error> {
let status = response.status();
let body = response
.text()
.unwrap_or_else(|error| format!("failed to read response: {error}"));
format!("{action} returned HTTP {status}: {body}").into()
}
fn is_matching_dev_revision(previous: &str, current: &str) -> bool {
let Some((previous_base, previous_hash)) = previous.rsplit_once("-dev.h") else {
return false;
};
let Some((current_base, current_hash)) = current.rsplit_once("-dev.h") else {
return false;
};
previous_base == current_base && valid_dev_hash(previous_hash) && valid_dev_hash(current_hash)
}
fn valid_dev_hash(value: &str) -> bool {
value.len() == 12 && value.bytes().all(|byte| byte.is_ascii_hexdigit())
}
fn watch_roots(manifest_path: &Path) -> Result<Vec<PathBuf>, Box<dyn std::error::Error>> {
let component_root = manifest_path
.parent()
.ok_or("Cargo manifest must have a parent directory")?
.to_owned();
let mut roots = BTreeSet::from([component_root.clone()]);
let module_path = component_root.join(MODULE_MANIFEST_FILE);
if module_path.is_file() {
let module = ModuleManifest::read(&module_path)?;
for replacement in module.replacements.values() {
let path = component_root.join(&replacement.path);
if path.exists() {
roots.insert(fs::canonicalize(path)?);
}
}
}
Ok(roots.into_iter().collect())
}
fn source_fingerprint(roots: &[PathBuf]) -> Result<Vec<u8>, Box<dyn std::error::Error>> {
let mut files = BTreeSet::new();
for root in roots {
collect_source_files(root, &mut files)?;
}
let mut digest = Sha256::new();
for path in files {
digest.update(path.to_string_lossy().as_bytes());
match fs::read(&path) {
Ok(bytes) => digest.update(bytes),
Err(error) if error.kind() == io::ErrorKind::NotFound => {}
Err(error) => return Err(error.into()),
}
}
Ok(digest.finalize().to_vec())
}
fn collect_source_files(
path: &Path,
files: &mut BTreeSet<PathBuf>,
) -> Result<(), Box<dyn std::error::Error>> {
if path.is_dir() {
if ignored_directory(path) {
return Ok(());
}
for entry in fs::read_dir(path)? {
collect_source_files(&entry?.path(), files)?;
}
} else if is_source_file(path) {
files.insert(path.to_owned());
}
Ok(())
}
fn ignored_directory(path: &Path) -> bool {
let name = path.file_name().and_then(|name| name.to_str());
if matches!(
name,
Some(".git" | ".wasmeld" | "deps" | "dist" | "node_modules" | "target")
) {
return name != Some("deps")
|| path
.parent()
.and_then(Path::file_name)
.and_then(|name| name.to_str())
== Some("wit");
}
false
}
fn is_source_file(path: &Path) -> bool {
matches!(
path.extension().and_then(|extension| extension.to_str()),
Some("rs" | "wit")
) || matches!(
path.file_name().and_then(|name| name.to_str()),
Some("Cargo.lock" | "Cargo.toml" | "config.toml" | "wasmeld.toml" | "wit.lock")
)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn recognizes_only_matching_hash_development_revisions() {
assert!(is_matching_dev_revision(
"0.1.0-dev.h0123456789ab",
"0.1.0-dev.habcdef012345"
));
assert!(!is_matching_dev_revision(
"0.2.0-dev.h0123456789ab",
"0.1.0-dev.habcdef012345"
));
assert!(!is_matching_dev_revision(
"0.1.0",
"0.1.0-dev.habcdef012345"
));
}
#[test]
fn fingerprints_sources_but_ignores_materialized_wit_dependencies() {
let root = tempfile::tempdir().unwrap();
let source = root.path().join("src/lib.rs");
let generated = root.path().join("wit/deps/example/package.wit");
fs::create_dir_all(source.parent().unwrap()).unwrap();
fs::create_dir_all(generated.parent().unwrap()).unwrap();
fs::write(&source, "pub fn first() {}").unwrap();
fs::write(&generated, "package generated:dependency@1.0.0;").unwrap();
let roots = vec![root.path().to_owned()];
let initial = source_fingerprint(&roots).unwrap();
fs::write(&generated, "package generated:dependency@2.0.0;").unwrap();
assert_eq!(source_fingerprint(&roots).unwrap(), initial);
fs::write(&source, "pub fn second() {}").unwrap();
assert_ne!(source_fingerprint(&roots).unwrap(), initial);
}
}
+108 -28
View File
@@ -1,5 +1,7 @@
//! Command-line entry point for Component packaging and WIT dependency workflows. //! Command-line entry point for Component packaging and WIT dependency workflows.
mod dev;
use std::{ use std::{
env, fs, env, fs,
path::{Path, PathBuf}, path::{Path, PathBuf},
@@ -7,6 +9,7 @@ use std::{
}; };
use serde::Deserialize; use serde::Deserialize;
use sha2::{Digest, Sha256};
use wasmeld_package::{ use wasmeld_package::{
PackageManifest, PackageManifest,
module::{MODULE_MANIFEST_FILE, ModuleManifest, find_module_manifest, sync_dependencies}, module::{MODULE_MANIFEST_FILE, ModuleManifest, find_module_manifest, sync_dependencies},
@@ -16,6 +19,41 @@ use wasmeld_package::{
const TARGET: &str = "wasm32-wasip2"; const TARGET: &str = "wasm32-wasip2";
#[derive(Clone, Copy)]
pub(crate) enum BuildProfile {
Debug,
Release,
}
impl BuildProfile {
fn directory(self) -> &'static str {
match self {
Self::Debug => "debug",
Self::Release => "release",
}
}
}
pub(crate) enum PackageIdentity {
Cargo,
Development { id: Option<String> },
}
pub(crate) struct PackRequest {
pub manifest_path: PathBuf,
pub output: Option<PathBuf>,
pub no_build: bool,
pub locked: bool,
pub profile: BuildProfile,
pub identity: PackageIdentity,
}
pub(crate) struct PackedComponent {
pub manifest: PackageManifest,
pub output: PathBuf,
pub artifact: PathBuf,
}
#[derive(Debug, Deserialize)] #[derive(Debug, Deserialize)]
struct CargoMetadata { struct CargoMetadata {
packages: Vec<CargoPackage>, packages: Vec<CargoPackage>,
@@ -51,6 +89,7 @@ fn run() -> Result<(), Box<dyn std::error::Error>> {
let mut arguments = env::args().skip(1); let mut arguments = env::args().skip(1);
match arguments.next().as_deref() { match arguments.next().as_deref() {
Some("pack") => run_pack(arguments.collect()), Some("pack") => run_pack(arguments.collect()),
Some("dev") => dev::run(arguments.collect()),
Some("wit") => run_wit(arguments.collect()), Some("wit") => run_wit(arguments.collect()),
_ => Err(usage().into()), _ => Err(usage().into()),
} }
@@ -78,69 +117,102 @@ fn run_pack(arguments: Vec<String>) -> Result<(), Box<dyn std::error::Error>> {
} }
} }
let manifest_path = fs::canonicalize(manifest_path)?; let packed = pack_component(PackRequest {
sync_component_dependencies(&manifest_path, locked)?; manifest_path: PathBuf::from(manifest_path),
output,
no_build,
locked,
profile: BuildProfile::Release,
identity: PackageIdentity::Cargo,
})?;
println!(
"packed {}@{} -> {}",
packed.manifest.id,
packed.manifest.revision,
packed.output.display()
);
println!("component: {}", packed.artifact.display());
println!("sha256: {}", packed.manifest.sha256);
Ok(())
}
pub(crate) fn pack_component(
request: PackRequest,
) -> Result<PackedComponent, Box<dyn std::error::Error>> {
let manifest_path = fs::canonicalize(request.manifest_path)?;
sync_component_dependencies(&manifest_path, request.locked)?;
let metadata = cargo_metadata(&manifest_path)?; let metadata = cargo_metadata(&manifest_path)?;
let package = metadata let package = metadata
.packages .packages
.iter() .iter()
.find(|package| package.manifest_path == manifest_path) .find(|package| package.manifest_path == manifest_path)
.ok_or("Cargo metadata did not contain the requested package")?; .ok_or("Cargo metadata did not contain the requested package")?;
let id = package let cargo_id = package
.metadata .metadata
.get("wasmeld") .get("wasmeld")
.and_then(|metadata| metadata.get("id")) .and_then(|metadata| metadata.get("id"))
.and_then(serde_json::Value::as_str) .and_then(serde_json::Value::as_str)
.ok_or("[package.metadata.wasmeld].id is required")?; .ok_or("[package.metadata.wasmeld].id is required")?
.to_owned();
let world = package let world = package
.metadata .metadata
.get("wasmeld") .get("wasmeld")
.and_then(|metadata| metadata.get("world")) .and_then(|metadata| metadata.get("world"))
.and_then(serde_json::Value::as_str) .and_then(serde_json::Value::as_str)
.ok_or("[package.metadata.wasmeld].world is required")?; .ok_or("[package.metadata.wasmeld].world is required")?
PackageManifest::new(id, &package.version, world, b"identity-validation").validate()?; .to_owned();
let cargo_revision = package.version.clone();
let target = package let target = package
.targets .targets
.iter() .iter()
.find(|target| target.crate_types.iter().any(|kind| kind == "cdylib")) .find(|target| target.crate_types.iter().any(|kind| kind == "cdylib"))
.ok_or("component package must expose a cdylib target")?; .ok_or("component package must expose a cdylib target")?
.name
.clone();
if !no_build { if !request.no_build {
build_component(&manifest_path)?; build_component(&manifest_path, request.profile)?;
} }
let artifact = metadata let artifact = metadata
.target_directory .target_directory
.join(TARGET) .join(TARGET)
.join("release") .join(request.profile.directory())
.join(format!("{}.wasm", target.name.replace('-', "_"))); .join(format!("{}.wasm", target.replace('-', "_")));
let component = fs::read(&artifact).map_err(|error| { let component = fs::read(&artifact).map_err(|error| {
format!( format!(
"failed to read built component {}: {error}", "failed to read built component {}: {error}",
artifact.display() artifact.display()
) )
})?; })?;
let output = output.unwrap_or_else(|| { let (id, revision) = match request.identity {
PackageIdentity::Cargo => (cargo_id, cargo_revision),
PackageIdentity::Development { id } => {
let digest = format!("{:x}", Sha256::digest(&component));
(
id.unwrap_or_else(|| format!("{cargo_id}-dev")),
format!("{cargo_revision}-dev.h{}", &digest[..12]),
)
}
};
PackageManifest::new(&id, &revision, &world, b"identity-validation").validate()?;
let output = request.output.unwrap_or_else(|| {
metadata metadata
.workspace_root .workspace_root
.join("dist") .join("dist")
.join(format!("{id}-{}.wasmpkg", package.version)) .join(format!("{id}-{revision}.wasmpkg"))
}); });
if let Some(parent) = output.parent() { if let Some(parent) = output.parent() {
fs::create_dir_all(parent)?; fs::create_dir_all(parent)?;
} }
let file = fs::File::create(&output)?; let file = fs::File::create(&output)?;
let manifest = write_package(file, id, &package.version, world, &component)?; let manifest = write_package(file, id, revision, world, &component)?;
Ok(PackedComponent {
println!( manifest,
"packed {}@{} -> {}", output,
manifest.id, artifact,
manifest.revision, })
output.display()
);
println!("component: {}", artifact.display());
println!("sha256: {}", manifest.sha256);
Ok(())
} }
fn run_wit(arguments: Vec<String>) -> Result<(), Box<dyn std::error::Error>> { fn run_wit(arguments: Vec<String>) -> Result<(), Box<dyn std::error::Error>> {
@@ -426,14 +498,21 @@ fn cargo_metadata(manifest_path: &Path) -> Result<CargoMetadata, Box<dyn std::er
Ok(serde_json::from_slice(&output.stdout)?) Ok(serde_json::from_slice(&output.stdout)?)
} }
fn build_component(manifest_path: &Path) -> Result<(), Box<dyn std::error::Error>> { fn build_component(
manifest_path: &Path,
profile: BuildProfile,
) -> Result<(), Box<dyn std::error::Error>> {
let toolchain = env::var("WASMELD_COMPONENT_TOOLCHAIN").unwrap_or_else(|_| "1.90.0".to_owned()); let toolchain = env::var("WASMELD_COMPONENT_TOOLCHAIN").unwrap_or_else(|_| "1.90.0".to_owned());
let status = Command::new("rustup") let mut command = Command::new("rustup");
command
.args(["run", &toolchain, "cargo", "build"]) .args(["run", &toolchain, "cargo", "build"])
.arg("--manifest-path") .arg("--manifest-path")
.arg(manifest_path) .arg(manifest_path)
.args(["--target", TARGET, "--release"]) .args(["--target", TARGET]);
.status()?; if matches!(profile, BuildProfile::Release) {
command.arg("--release");
}
let status = command.status()?;
if !status.success() { if !status.success() {
return Err(format!("component build failed with status {status}").into()); return Err(format!("component build failed with status {status}").into());
} }
@@ -443,6 +522,7 @@ fn build_component(manifest_path: &Path) -> Result<(), Box<dyn std::error::Error
fn usage() -> String { fn usage() -> String {
"usage: "usage:
wasmeld pack <Cargo.toml> [--output <file.wasmpkg>] [--no-build] [--locked] wasmeld pack <Cargo.toml> [--output <file.wasmpkg>] [--no-build] [--locked]
wasmeld dev <Cargo.toml> [--console <url>] [--id <service-id>] [--locked] [--release] [--once] [--poll-ms <milliseconds>]
wasmeld wit fetch [--manifest <wasmeld.toml>] [--locked] wasmeld wit fetch [--manifest <wasmeld.toml>] [--locked]
wasmeld wit tidy [--manifest <wasmeld.toml>] [--locked] wasmeld wit tidy [--manifest <wasmeld.toml>] [--locked]
wasmeld wit graph [--manifest <wasmeld.toml>] wasmeld wit graph [--manifest <wasmeld.toml>]
+145
View File
@@ -0,0 +1,145 @@
//! `wasmeld:kv/store@0.1.0` host implementation and backend boundary.
use std::{fmt, sync::Arc, time::Duration};
use thiserror::Error;
use wasmtime::component::Linker;
use crate::{RuntimeError, runtime::HostState};
wasmtime::component::bindgen!({
path: "../../wit/kv",
world: "kv-host",
});
pub(crate) const INTERFACE: &str = "wasmeld:kv/store@0.1.0";
pub(crate) const PACKAGE: &str = "wasmeld:kv";
pub(crate) const NAME: &str = "store";
pub(crate) const VERSION: &str = "0.1.0";
/// Maximum UTF-8 byte length accepted for one KV key.
pub const KV_MAX_KEY_BYTES: usize = 256;
/// Maximum byte length accepted for one KV value.
pub const KV_MAX_VALUE_BYTES: usize = 64 * 1024;
/// Synchronous storage boundary used by the Wasmtime host implementation.
///
/// Implementations must bound their own blocking time because calls execute on
/// the Component Actor thread.
pub trait KvBackend: fmt::Debug + Send + Sync {
/// Returns the value stored under a service-scoped key.
fn get(
&self,
service_id: &str,
key: &str,
timeout: Duration,
) -> Result<Option<Vec<u8>>, KvBackendError>;
/// Creates or replaces a service-scoped value.
fn set(
&self,
service_id: &str,
key: &str,
value: &[u8],
timeout: Duration,
) -> Result<(), KvBackendError>;
/// Removes a service-scoped value if it exists.
fn delete(&self, service_id: &str, key: &str, timeout: Duration) -> Result<(), KvBackendError>;
}
/// Storage failures returned by a [`KvBackend`].
#[derive(Debug, Error)]
pub enum KvBackendError {
#[error("KV backend is busy")]
Busy,
#[error("KV backend timed out")]
Timeout,
#[error("KV backend operation failed: {0}")]
Operation(String),
}
pub(crate) struct KvState {
service_id: String,
backend: Arc<dyn KvBackend>,
timeout: Duration,
}
impl KvState {
pub(crate) fn new(service_id: &str, backend: Arc<dyn KvBackend>, timeout: Duration) -> Self {
Self {
service_id: service_id.to_owned(),
backend,
timeout,
}
}
fn get(&self, key: &str) -> Result<Option<Vec<u8>>, wasmeld::kv::store::KvError> {
validate_key(key)?;
self.backend
.get(&self.service_id, key, self.timeout)
.map_err(operation_failed)
.and_then(|value| match value {
Some(value) if value.len() > KV_MAX_VALUE_BYTES => {
Err(wasmeld::kv::store::KvError::ValueTooLarge(
u64::try_from(value.len()).unwrap_or(u64::MAX),
))
}
value => Ok(value),
})
}
fn set(&self, key: &str, value: &[u8]) -> Result<(), wasmeld::kv::store::KvError> {
validate_key(key)?;
if value.len() > KV_MAX_VALUE_BYTES {
return Err(wasmeld::kv::store::KvError::ValueTooLarge(
u64::try_from(value.len()).unwrap_or(u64::MAX),
));
}
self.backend
.set(&self.service_id, key, value, self.timeout)
.map_err(operation_failed)
}
fn delete(&self, key: &str) -> Result<(), wasmeld::kv::store::KvError> {
validate_key(key)?;
self.backend
.delete(&self.service_id, key, self.timeout)
.map_err(operation_failed)
}
}
impl wasmeld::kv::store::Host for HostState {
fn get(&mut self, key: String) -> Result<Option<Vec<u8>>, wasmeld::kv::store::KvError> {
self.capabilities.kv().get(&key)
}
fn set(&mut self, key: String, value: Vec<u8>) -> Result<(), wasmeld::kv::store::KvError> {
self.capabilities.kv().set(&key, &value)
}
fn delete(&mut self, key: String) -> Result<(), wasmeld::kv::store::KvError> {
self.capabilities.kv().delete(&key)
}
}
pub(crate) fn add_to_linker(linker: &mut Linker<HostState>) -> Result<(), RuntimeError> {
wasmeld::kv::store::add_to_linker::<HostState, HostState>(linker, |state| state)
.map_err(|error| RuntimeError::ActorInitialization(error.to_string()))
}
fn validate_key(key: &str) -> Result<(), wasmeld::kv::store::KvError> {
if key.is_empty() {
return Err(wasmeld::kv::store::KvError::InvalidKey(
"key must not be empty".to_owned(),
));
}
if key.len() > KV_MAX_KEY_BYTES {
return Err(wasmeld::kv::store::KvError::InvalidKey(format!(
"key exceeds the {KV_MAX_KEY_BYTES}-byte limit"
)));
}
Ok(())
}
fn operation_failed(error: KvBackendError) -> wasmeld::kv::store::KvError {
wasmeld::kv::store::KvError::OperationFailed(error.to_string())
}
+22 -1
View File
@@ -1,22 +1,37 @@
//! Versioned host capabilities available to WebAssembly Components. //! Versioned host capabilities available to WebAssembly Components.
mod clock; mod clock;
mod kv;
mod registry; mod registry;
pub use kv::{KV_MAX_KEY_BYTES, KV_MAX_VALUE_BYTES, KvBackend, KvBackendError};
pub use registry::CapabilityDescriptor; pub use registry::CapabilityDescriptor;
pub(crate) use registry::{Capability, CapabilityRegistry}; pub(crate) use registry::{Capability, CapabilityRegistry};
/// Per-Actor state owned by enabled host capabilities. /// Per-Actor state owned by enabled host capabilities.
pub(crate) struct HostCapabilities { pub(crate) struct HostCapabilities {
clock: Option<clock::ClockState>, clock: Option<clock::ClockState>,
kv: Option<kv::KvState>,
} }
impl HostCapabilities { impl HostCapabilities {
pub(crate) fn new(capabilities: &[Capability]) -> Self { pub(crate) fn new(
capabilities: &[Capability],
service_id: &str,
kv_backend: Option<std::sync::Arc<dyn KvBackend>>,
deadline: std::time::Duration,
) -> Self {
Self { Self {
clock: capabilities clock: capabilities
.contains(&Capability::MonotonicClock) .contains(&Capability::MonotonicClock)
.then(clock::ClockState::new), .then(clock::ClockState::new),
kv: capabilities.contains(&Capability::KvStore).then(|| {
kv::KvState::new(
service_id,
kv_backend.expect("KV Components are rejected when no backend is configured"),
deadline,
)
}),
} }
} }
@@ -26,4 +41,10 @@ impl HostCapabilities {
.expect("the Clock interface is linked only when its state is enabled") .expect("the Clock interface is linked only when its state is enabled")
.now() .now()
} }
fn kv(&self) -> &kv::KvState {
self.kv
.as_ref()
.expect("the KV interface is linked only when its state is enabled")
}
} }
@@ -2,7 +2,7 @@
use wasmtime::component::Linker; use wasmtime::component::Linker;
use super::clock; use super::{clock, kv};
use crate::{RuntimeError, runtime::HostState}; use crate::{RuntimeError, runtime::HostState};
/// Stable description of one versioned WIT interface used by a Component. /// Stable description of one versioned WIT interface used by a Component.
@@ -53,6 +53,7 @@ impl CapabilityDescriptor {
#[derive(Clone, Copy, Debug, Eq, Ord, PartialEq, PartialOrd)] #[derive(Clone, Copy, Debug, Eq, Ord, PartialEq, PartialOrd)]
pub(crate) enum Capability { pub(crate) enum Capability {
MonotonicClock, MonotonicClock,
KvStore,
} }
impl Capability { impl Capability {
@@ -64,12 +65,16 @@ impl Capability {
clock::NAME, clock::NAME,
clock::VERSION, clock::VERSION,
), ),
Self::KvStore => {
CapabilityDescriptor::new(kv::INTERFACE, kv::PACKAGE, kv::NAME, kv::VERSION)
}
} }
} }
pub(crate) fn add_to_linker(self, linker: &mut Linker<HostState>) -> Result<(), RuntimeError> { pub(crate) fn add_to_linker(self, linker: &mut Linker<HostState>) -> Result<(), RuntimeError> {
match self { match self {
Self::MonotonicClock => clock::add_to_linker(linker), Self::MonotonicClock => clock::add_to_linker(linker),
Self::KvStore => kv::add_to_linker(linker),
} }
} }
} }
@@ -82,6 +87,7 @@ impl CapabilityRegistry {
pub(crate) fn resolve(interface: &str) -> Option<Capability> { pub(crate) fn resolve(interface: &str) -> Option<Capability> {
match interface { match interface {
clock::INTERFACE => Some(Capability::MonotonicClock), clock::INTERFACE => Some(Capability::MonotonicClock),
kv::INTERFACE => Some(Capability::KvStore),
_ => None, _ => None,
} }
} }
@@ -116,5 +122,10 @@ mod tests {
CapabilityRegistry::resolve("wasmeld:clock/other@0.1.0"), CapabilityRegistry::resolve("wasmeld:clock/other@0.1.0"),
None None
); );
assert_eq!(
CapabilityRegistry::resolve("wasmeld:kv/store@0.1.0"),
Some(Capability::KvStore)
);
assert_eq!(CapabilityRegistry::resolve("wasmeld:kv/store@0.2.0"), None);
} }
} }
+3
View File
@@ -30,6 +30,9 @@ pub enum RuntimeError {
#[error("component imports unsupported capability {0}")] #[error("component imports unsupported capability {0}")]
UnsupportedImport(String), UnsupportedImport(String),
#[error("component requires unavailable capability {0}")]
CapabilityUnavailable(String),
#[error("runtime creation failed: {0}")] #[error("runtime creation failed: {0}")]
RuntimeCreation(#[source] wasmtime::Error), RuntimeCreation(#[source] wasmtime::Error),
+3 -1
View File
@@ -11,7 +11,9 @@ mod error;
mod manifest; mod manifest;
mod runtime; mod runtime;
pub use capability::CapabilityDescriptor; pub use capability::{
CapabilityDescriptor, KV_MAX_KEY_BYTES, KV_MAX_VALUE_BYTES, KvBackend, KvBackendError,
};
pub use error::RuntimeError; pub use error::RuntimeError;
pub use manifest::{ResourceLimits, ServiceKey, ServiceManifest}; pub use manifest::{ResourceLimits, ServiceKey, ServiceManifest};
pub use runtime::{ActorHandle, Runtime, RuntimeConfig, RuntimeStats}; pub use runtime::{ActorHandle, Runtime, RuntimeConfig, RuntimeStats};
+80 -5
View File
@@ -30,7 +30,7 @@ use wasmtime_wasi::{
}; };
use crate::{ use crate::{
CapabilityDescriptor, RuntimeError, ServiceKey, ServiceManifest, CapabilityDescriptor, KvBackend, RuntimeError, ServiceKey, ServiceManifest,
bindings::{ServiceComponent, ServiceError}, bindings::{ServiceComponent, ServiceError},
capability::{Capability, CapabilityRegistry, HostCapabilities}, capability::{Capability, CapabilityRegistry, HostCapabilities},
manifest::ResourceLimits, manifest::ResourceLimits,
@@ -61,6 +61,8 @@ pub struct RuntimeConfig {
pub epoch_tick: Duration, pub epoch_tick: Duration,
/// Maximum native stack reservation for WebAssembly execution. /// Maximum native stack reservation for WebAssembly execution.
pub max_wasm_stack: usize, pub max_wasm_stack: usize,
/// Service-scoped storage used by Components importing the KV capability.
pub kv_backend: Option<Arc<dyn KvBackend>>,
} }
impl Default for RuntimeConfig { impl Default for RuntimeConfig {
@@ -68,6 +70,7 @@ impl Default for RuntimeConfig {
Self { Self {
epoch_tick: DEFAULT_EPOCH_TICK, epoch_tick: DEFAULT_EPOCH_TICK,
max_wasm_stack: DEFAULT_MAX_WASM_STACK, max_wasm_stack: DEFAULT_MAX_WASM_STACK,
kv_backend: None,
} }
} }
} }
@@ -117,6 +120,7 @@ struct RuntimeInner {
actors: Mutex<HashMap<ServiceKey, ActorHandle>>, actors: Mutex<HashMap<ServiceKey, ActorHandle>>,
lifecycle: Mutex<()>, lifecycle: Mutex<()>,
ticker: Arc<EpochTicker>, ticker: Arc<EpochTicker>,
kv_backend: Option<Arc<dyn KvBackend>>,
} }
struct EpochTicker { struct EpochTicker {
@@ -189,6 +193,7 @@ impl Runtime {
actors: Mutex::new(HashMap::new()), actors: Mutex::new(HashMap::new()),
lifecycle: Mutex::new(()), lifecycle: Mutex::new(()),
ticker, ticker,
kv_backend: config.kv_backend,
}), }),
}) })
} }
@@ -207,6 +212,11 @@ impl Runtime {
let component = Component::new(&self.inner.engine, component_bytes.as_ref()) let component = Component::new(&self.inner.engine, component_bytes.as_ref())
.map_err(RuntimeError::ComponentCompilation)?; .map_err(RuntimeError::ComponentCompilation)?;
let capabilities = validate_component_imports(&component, &self.inner.engine)?; let capabilities = validate_component_imports(&component, &self.inner.engine)?;
if capabilities.contains(&Capability::KvStore) && self.inner.kv_backend.is_none() {
return Err(RuntimeError::CapabilityUnavailable(
Capability::KvStore.descriptor().interface().to_owned(),
));
}
let mut services = self let mut services = self
.inner .inner
@@ -298,6 +308,7 @@ impl Runtime {
key.clone(), key.clone(),
service, service,
init_config, init_config,
self.inner.kv_backend.clone(),
)?; )?;
self.inner self.inner
@@ -379,6 +390,55 @@ impl Runtime {
self.start(&key, init_config) self.start(&key, init_config)
} }
/// Stops and removes one registered service revision.
///
/// Callers must switch any external deployment reference away from this
/// revision before unregistering it.
pub fn unregister(&self, key: &ServiceKey) -> Result<(), RuntimeError> {
let _lifecycle = self
.inner
.lifecycle
.lock()
.map_err(|_| RuntimeError::LockPoisoned)?;
if !self
.inner
.services
.lock()
.map_err(|_| RuntimeError::LockPoisoned)?
.contains_key(key)
{
return Err(RuntimeError::ServiceNotRegistered(key.clone()));
}
let actor = self
.inner
.actors
.lock()
.map_err(|_| RuntimeError::LockPoisoned)?
.get(key)
.cloned();
if let Some(actor) = actor {
if actor.is_available() {
actor.stop()?;
} else if actor.is_alive() {
return Err(RuntimeError::ActorUnavailable(key.clone()));
}
self.inner
.actors
.lock()
.map_err(|_| RuntimeError::LockPoisoned)?
.remove(key);
}
self.inner
.services
.lock()
.map_err(|_| RuntimeError::LockPoisoned)?
.remove(key);
Ok(())
}
/// Returns counts without exposing internal Engine or Store state. /// Returns counts without exposing internal Engine or Store state.
pub fn stats(&self) -> Result<RuntimeStats, RuntimeError> { pub fn stats(&self) -> Result<RuntimeStats, RuntimeError> {
let registered_services = self let registered_services = self
@@ -580,7 +640,12 @@ pub(crate) struct HostState {
} }
impl HostState { impl HostState {
fn new(limits: &ResourceLimits, capabilities: &[Capability]) -> Self { fn new(
limits: &ResourceLimits,
capabilities: &[Capability],
service_id: &str,
kv_backend: Option<Arc<dyn KvBackend>>,
) -> Self {
let store_limits = StoreLimitsBuilder::new() let store_limits = StoreLimitsBuilder::new()
.memory_size(limits.memory_bytes) .memory_size(limits.memory_bytes)
.table_elements(ACTOR_TABLE_ELEMENT_LIMIT) .table_elements(ACTOR_TABLE_ELEMENT_LIMIT)
@@ -600,7 +665,12 @@ impl HostState {
store_limits, store_limits,
wasi: wasi.build(), wasi: wasi.build(),
wasi_table: ResourceTable::new(), wasi_table: ResourceTable::new(),
capabilities: HostCapabilities::new(capabilities), capabilities: HostCapabilities::new(
capabilities,
service_id,
kv_backend,
limits.deadline(),
),
} }
} }
} }
@@ -639,6 +709,7 @@ impl ActorWorker {
key: ServiceKey, key: ServiceKey,
service: RegisteredService, service: RegisteredService,
init_config: Vec<u8>, init_config: Vec<u8>,
kv_backend: Option<Arc<dyn KvBackend>>,
) -> Result<Self, RuntimeError> { ) -> Result<Self, RuntimeError> {
let limits = Arc::new(service.manifest.limits.clone()); let limits = Arc::new(service.manifest.limits.clone());
if init_config.len() > limits.max_input_bytes { if init_config.len() > limits.max_input_bytes {
@@ -648,7 +719,10 @@ impl ActorWorker {
}); });
} }
let mut store = Store::new(&engine, HostState::new(&limits, &service.capabilities)); let mut store = Store::new(
&engine,
HostState::new(&limits, &service.capabilities, key.id(), kv_backend),
);
store.limiter(|state| &mut state.store_limits); store.limiter(|state| &mut state.store_limits);
let mut linker = Linker::new(&engine); let mut linker = Linker::new(&engine);
@@ -797,6 +871,7 @@ fn spawn_actor(
key: ServiceKey, key: ServiceKey,
service: RegisteredService, service: RegisteredService,
init_config: Vec<u8>, init_config: Vec<u8>,
kv_backend: Option<Arc<dyn KvBackend>>,
) -> Result<ActorHandle, RuntimeError> { ) -> Result<ActorHandle, RuntimeError> {
let limits = Arc::new(service.manifest.limits.clone()); let limits = Arc::new(service.manifest.limits.clone());
let (sender, receiver) = mpsc::sync_channel(limits.mailbox_capacity); let (sender, receiver) = mpsc::sync_channel(limits.mailbox_capacity);
@@ -811,7 +886,7 @@ fn spawn_actor(
.spawn(move || { .spawn(move || {
let _ticker = worker_ticker; let _ticker = worker_ticker;
match ActorWorker::create(engine, epoch_tick, key, service, init_config) { match ActorWorker::create(engine, epoch_tick, key, service, init_config, kv_backend) {
Ok(mut worker) => { Ok(mut worker) => {
worker_status.mark_ready(); worker_status.mark_ready();
let _ = ready_sender.send(Ok(())); let _ = ready_sender.send(Ok(()));
@@ -1,13 +1,17 @@
use std::{ use std::{
collections::HashMap,
fs, fs,
path::{Path, PathBuf}, path::{Path, PathBuf},
process::Command, process::Command,
sync::OnceLock, sync::{Arc, Mutex, OnceLock},
thread, thread,
time::{Duration, Instant}, time::{Duration, Instant},
}; };
use wasmeld_runtime::{ResourceLimits, Runtime, RuntimeConfig, RuntimeError, ServiceManifest}; use wasmeld_runtime::{
KV_MAX_VALUE_BYTES, KvBackend, KvBackendError, ResourceLimits, Runtime, RuntimeConfig,
RuntimeError, ServiceManifest,
};
static COMPONENTS_BUILT: OnceLock<()> = OnceLock::new(); static COMPONENTS_BUILT: OnceLock<()> = OnceLock::new();
@@ -188,6 +192,114 @@ fn explicit_clock_capability_is_linked() {
runtime.stop(&key).unwrap(); runtime.stop(&key).unwrap();
} }
#[test]
fn kv_capability_is_service_scoped_and_shared_across_revisions() {
let backend = Arc::new(MemoryKvBackend::default());
let runtime = Runtime::new(RuntimeConfig {
kv_backend: Some(backend),
..RuntimeConfig::default()
})
.expect("runtime should start");
let first_key = register_component_revision(
&runtime,
"shared-kv",
"0.1.0",
"kv_probe_component.wasm",
test_limits(),
);
let second_key = register_component_revision(
&runtime,
"shared-kv",
"0.2.0",
"kv_probe_component.wasm",
test_limits(),
);
let isolated_key = register_component_revision(
&runtime,
"isolated-kv",
"0.1.0",
"kv_probe_component.wasm",
test_limits(),
);
let first = runtime.start(&first_key, Vec::new()).unwrap();
let second = runtime.start(&second_key, Vec::new()).unwrap();
let isolated = runtime.start(&isolated_key, Vec::new()).unwrap();
assert_eq!(first.invoke(b"set:answer:forty-two".to_vec()).unwrap(), b"");
assert_eq!(second.invoke(b"get:answer".to_vec()).unwrap(), b"forty-two");
assert_eq!(isolated.invoke(b"get:answer".to_vec()).unwrap(), b"");
assert_eq!(
runtime.capabilities(&first_key).unwrap()[0].interface(),
"wasmeld:kv/store@0.1.0"
);
runtime.stop_all().unwrap();
}
#[test]
fn kv_capability_requires_a_backend_and_enforces_limits() {
let artifact = component_artifact("kv_probe_component.wasm");
let manifest = ServiceManifest {
id: "kv-without-backend".to_owned(),
revision: "0.1.0".to_owned(),
component: artifact.display().to_string(),
world: component_world("kv_probe_component.wasm").to_owned(),
limits: test_limits(),
};
let bytes = fs::read(&artifact).unwrap();
let runtime = Runtime::new(RuntimeConfig::default()).unwrap();
assert!(matches!(
runtime.register(manifest, &bytes),
Err(RuntimeError::CapabilityUnavailable(interface))
if interface == "wasmeld:kv/store@0.1.0"
));
let runtime = Runtime::new(RuntimeConfig {
kv_backend: Some(Arc::new(MemoryKvBackend::default())),
..RuntimeConfig::default()
})
.unwrap();
let mut limits = test_limits();
limits.max_input_bytes = 128 * 1024;
let key = register_component(&runtime, "kv-limits", "kv_probe_component.wasm", limits);
let actor = runtime.start(&key, Vec::new()).unwrap();
let long_key = format!("get:{}", "k".repeat(257));
assert!(matches!(
actor.invoke(long_key.into_bytes()),
Err(RuntimeError::ComponentError { message, .. })
if message.contains("InvalidKey")
));
let mut oversized = b"set:key:".to_vec();
oversized.resize(oversized.len() + KV_MAX_VALUE_BYTES + 1, b'x');
assert!(matches!(
actor.invoke(oversized),
Err(RuntimeError::ComponentError { message, .. })
if message.contains("ValueTooLarge")
));
runtime.stop(&key).unwrap();
}
#[test]
fn unregister_stops_actor_and_releases_compiled_revision() {
let runtime = Runtime::new(RuntimeConfig::default()).expect("runtime should start");
let key = register_component(&runtime, "unregister", "echo_component.wasm", test_limits());
runtime.start(&key, Vec::new()).unwrap();
runtime.unregister(&key).unwrap();
let stats = runtime.stats().unwrap();
assert_eq!(stats.registered_services, 0);
assert_eq!(stats.running_actors, 0);
assert!(matches!(
runtime.start(&key, Vec::new()),
Err(RuntimeError::ServiceNotRegistered(missing)) if missing == key
));
}
#[test] #[test]
fn registration_rejects_non_whitelisted_wasi_imports() { fn registration_rejects_non_whitelisted_wasi_imports() {
let runtime = Runtime::new(RuntimeConfig::default()).expect("runtime should start"); let runtime = Runtime::new(RuntimeConfig::default()).expect("runtime should start");
@@ -225,11 +337,21 @@ fn register_component(
service_id: &str, service_id: &str,
artifact_name: &str, artifact_name: &str,
limits: ResourceLimits, limits: ResourceLimits,
) -> wasmeld_runtime::ServiceKey {
register_component_revision(runtime, service_id, "0.1.0", artifact_name, limits)
}
fn register_component_revision(
runtime: &Runtime,
service_id: &str,
revision: &str,
artifact_name: &str,
limits: ResourceLimits,
) -> wasmeld_runtime::ServiceKey { ) -> wasmeld_runtime::ServiceKey {
let artifact = component_artifact(artifact_name); let artifact = component_artifact(artifact_name);
let manifest = ServiceManifest { let manifest = ServiceManifest {
id: service_id.to_owned(), id: service_id.to_owned(),
revision: "0.1.0".to_owned(), revision: revision.to_owned(),
component: artifact.display().to_string(), component: artifact.display().to_string(),
world: component_world(artifact_name).to_owned(), world: component_world(artifact_name).to_owned(),
limits, limits,
@@ -256,6 +378,7 @@ fn build_components_once() {
"components/fault/Cargo.toml", "components/fault/Cargo.toml",
"components/spin/Cargo.toml", "components/spin/Cargo.toml",
"components/clock-probe/Cargo.toml", "components/clock-probe/Cargo.toml",
"components/kv-probe/Cargo.toml",
"components/wasi-clock-probe/Cargo.toml", "components/wasi-clock-probe/Cargo.toml",
] { ] {
let status = Command::new("rustup") let status = Command::new("rustup")
@@ -323,9 +446,58 @@ fn component_world(artifact_name: &str) -> &'static str {
"fault_component.wasm" => "component:fault/fault-component@0.1.0", "fault_component.wasm" => "component:fault/fault-component@0.1.0",
"spin_component.wasm" => "component:spin/spin-component@0.1.0", "spin_component.wasm" => "component:spin/spin-component@0.1.0",
"clock_probe_component.wasm" => "component:clock-probe/clock-probe-component@0.1.0", "clock_probe_component.wasm" => "component:clock-probe/clock-probe-component@0.1.0",
"kv_probe_component.wasm" => "component:kv-probe/kv-probe-component@0.1.0",
"wasi_clock_probe_component.wasm" => { "wasi_clock_probe_component.wasm" => {
"component:wasi-clock-probe/wasi-clock-probe-component@0.1.0" "component:wasi-clock-probe/wasi-clock-probe-component@0.1.0"
} }
other => panic!("no WIT world registered for {other}"), other => panic!("no WIT world registered for {other}"),
} }
} }
#[derive(Debug, Default)]
struct MemoryKvBackend {
entries: Mutex<HashMap<(String, String), Vec<u8>>>,
}
impl KvBackend for MemoryKvBackend {
fn get(
&self,
service_id: &str,
key: &str,
_timeout: Duration,
) -> Result<Option<Vec<u8>>, KvBackendError> {
Ok(self
.entries
.lock()
.unwrap()
.get(&(service_id.to_owned(), key.to_owned()))
.cloned())
}
fn set(
&self,
service_id: &str,
key: &str,
value: &[u8],
_timeout: Duration,
) -> Result<(), KvBackendError> {
self.entries
.lock()
.unwrap()
.insert((service_id.to_owned(), key.to_owned()), value.to_vec());
Ok(())
}
fn delete(
&self,
service_id: &str,
key: &str,
_timeout: Duration,
) -> Result<(), KvBackendError> {
self.entries
.lock()
.unwrap()
.remove(&(service_id.to_owned(), key.to_owned()));
Ok(())
}
}
+17
View File
@@ -0,0 +1,17 @@
package wasmeld:kv@0.1.0;
interface store {
variant kv-error {
invalid-key(string),
value-too-large(u64),
operation-failed(string),
}
get: func(key: string) -> result<option<list<u8>>, kv-error>;
set: func(key: string, value: list<u8>) -> result<_, kv-error>;
delete: func(key: string) -> result<_, kv-error>;
}
world kv-host {
import store;
}