How to build Fastly Compute@Edge Services in C++
There are a lot of advantages to running code on CDN edge nodes: The latency from the user to your edge service is typically very very low. The services are generally stateless – and so can scale with demand very quickly. You don’t need your own servers. Uptime is very high.
But the platforms currently are very constrained with very limited speed and memory.
Most of the edge computing solutions run things like JavaScript. Given the relatively limited resources for edge computing, you cannot do anything computationally intensive with JS edge computing solutions. If you want to do anything interesting the only systems powerful enough user WebAssembly.
WebAssembly is compiled code that runs on at near native speeds (about 80% of native code in my tests).
There are two CDN providers offering WebAssembly – Fastly and Cloudflare. Between the two of them, Fastly is the more powerful, and easiest to use.
Fastly allows your WebAssembly package to be up to 100MB of compiled code and data. You can run up to 100ms of CPU time. This is enough to do pretty interesting things if you can use an efficient language.
There are a number of code generators for WebAssembly and one of them is Clang. Any language that has a Clang front end, can be connected to the WebAssembly code generator, and this includes efficient languages like C and C++.
Fastly provides dev kits for JavaScript, Rust and Go. But not for C++.
To make a compute@edge service involves more than just generating a WASM binary – but you also have to be able to make API calls into the compute@edge system. At minimum a compute@edge service must make API calls to read an http request and to answer it.
Fastly provides a WebAssembly ABI to connect with their system. That is, Fastly specifies the interface at a WebAssembly binary level. It is up to you to figure out how to express that ABI as an API in a target language. This is not easy.
Building a Fastly Compatible WASM Module
The first step is to install the WASI SDK. WASI is the “WebAssembly System Interface” this includes the tools to build WASM binaries and a minimal c like system interface. Think of WASM as set of opcodes and a binary executable format, similar to ARM32 or AMD64. WASI is the a basic runtime environment for WASM binaries.
Installing WASI-SDK installs c compiler tool chain and a standard c library – except that the WASI system libraries are much simpler. They have no native support for files, network, threads etc.
You install it on Debian like this:
# Update and install necessary tools sudo apt update sudo apt install wget tar # Download the latest WASI SDK (replace version as needed) wget https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-20/wasi-sdk-20.0-linux.tar.gz # Extract the WASI SDK tarball tar -xvzf wasi-sdk-20.0-linux.tar.gz # Move it to a system-wide directory (optional) sudo mv wasi-sdk-20 /opt/wasi-sdk # Add WASI SDK to PATH (temporary for current session) export WASI_SDK_PATH=/opt/wasi-sdk export PATH=$WASI_SDK_PATH/bin:$PATH # Add to shell configuration for persistence (e.g., .bashrc or .zshrc) echo 'export WASI_SDK_PATH=/opt/wasi-sdk' &> ~/.bashrc echo 'export PATH=$WASI_SDK_PATH/bin:$PATH' &>>; ~/.bashrc source ~/.bashrc # Verify installation clang --version wasm-ld --version
You build programs much like you would any c or c++ program. This is not that different than building for example a c++ program in Emscripten, except that WASI is much more constrained.
The main function looks like this, with no arguments:
int main() { return 0; }
You can run WASI binaries using wasmtime, the WASM runtime environment. That is, WASMTIME is a program that can be used to run your WASM binary on your local operating system.
A good first mini project is to write a small test program in WASM, compile it and run it with wasmtime:
wasmtime --dir=. hello-world.wasm -q
Making a Header File for the Fastly ABI
Fastly will run your WASM module inside a type of custom runtime enviroment, similar to wasmtime, but with additional custom functions. When your process runs, the fastly environment will take care of connecting to the internet, terminating SSL for you, sending and receiving http requests etc.
For example, to read an http request, you might do something like:
FastlyHttpReqHandle qhdl = 0; FastlyHttpBodyHandle bhdl = 0; FastlyStatus stat; // allocate a request handle if (( stat = fastly_http_req_body_downstream_get( qhdl, bhdl ))) return -1; char reqMethod[64]; char reqUri[1024]; u32 numRead; // read the request method if (( stat = fastly_http_req_method_get( qhdl, reqMethod, 64, numRead ))) return -1; reqMethod[numRead] = '\0'; // read the request uri if (( stat = fastly_http_req_uri_get( qhdl, reqUri, 1024, numRead ))) return -1; reqUri[numRead] = '\0';
But there isn’t a header file that defines these functions. You have to write it yourself from the ABI.
I wasn’t able to find a single definitive reference for the Fastly AB, but I used the following files as a guide:
- Rust Reference: https://docs.rs/fastly/latest/fastly/http/struct.Body.html
- assembly script reference: https://www.npmjs.com/package/@fastly/as-compute?activeTab=code
- witx reference: https://github.com/fastly/Viceroy/blob/main/lib/compute-at-edge-abi/compute-at-edge.witx
From these I built the following header file:
// These macros will save us a lot of verbiage later #define KX_STRINGIZE(X) KX_XSTRINGIZE(X) #define KX_XSTRINGIZE(X) #X #define F_DEF( MODULE, FUNCTION ) __attribute__((import_module( KX_STRINGIZE( MODULE )), import_name( KX_STRINGIZE( FUNCTION )))) FastlyStatus MODULE##_##FUNCTION // ------- Define necessary constants used in the ABI ------ enum FastlyStatus : u32 { FASTLY_OK, FASTLY_GENERIC_ERROR, FASTLY_INVALID_ARGUMENT, FASTLY_BAD_HANDLE, FASTLY_BUFFER_LEN, FASTLY_UNSUPPORTED, FASTLY_BAD_ALIGN, FASTLY_HTTP_INVALID, FASTLY_HTTP_USER, FASTLY_HTTP_INCOMPLETE, FASTLY_OPTIONAL_NONE, FASTLY_HTTP_HEAD_TOO_LARGE, FASTLY_HTTP_INVALID_STATUS, FASTLY_LIMIT_EXCEEDED }; enum FastlyHttpVersion : u32 { FASTLY_HTTP_0_9, FASTLY_HTTP_1_0, FASTLY_HTTP_1_1, FASTLY_HTTP_2, FASTLY_HTTP_3, }; enum FastlyFramingMode : u32 { FASTLY_FRAMING_AUTOMATIC, FASTLY_FRAMING_FROM_HEADERS }; enum FastlyKeepAliveMode : u32 { FASTLY_KEEPALIVE_AUTOMATIC, FASTLY_KEEPALIVE_NONE }; enum FastlyWriteEnd : u32 { FASTLY_WRITE_END_BACK, FASTLY_WRITE_END_FRONT, }; enum FastlyCertVerifyResult : u32 { FASTLY_CERT_VERIFY_OK, FASTLY_CERT_VERIFY_BAD_CERTIFICATE, FASTLY_CERT_VERIFY_CERTIFICATE_REVOKED, FASTLY_CERT_VERIFY_CERTIFICATE_EXPIRED, FASTLY_CERT_VERIFY_UNKNOWN_CA, FASTLY_CERT_VERIFY_CERTIFICATE_MISSING, FASTLY_CERT_VERIFY_CERTIFICATE_UNKNOWN }; enum FastlyCacheOverrideTag : u32 { FASTLY_CACHE_OVERRIDE_PASS = 0x00, FASTLY_CACHE_OVERRIDE_TTL = 0x01, FASTLY_CACHE_OVERRIDE_STALE_WHILE_REVALIDATE = 0x02, FASTLY_CACHE_OVERRIDE_PCI = 0x04, }; typedef s32 FastlyHttpBodyHandle; typedef s32 FastlyHttpReqHandle; typedef s32 FastlyHttpRespHandle; typedef s32 FastlyLogHandle; typedef u16 FastlyHttpStatus; extern "C" { // ---------------- HttpBody ------------------------------------------------------------------------------- F_DEF( fastly_http_body, new ) ( FastlyHttpBodyHandle& hdl ); F_DEF( fastly_http_body, append ) ( FastlyHttpBodyHandle dst, FastlyHttpBodyHandle src ); F_DEF( fastly_http_body, read ) ( FastlyHttpBodyHandle hdl, const char* buf, u32 bufLen, u32& numRead ); F_DEF( fastly_http_body, write ) ( FastlyHttpBodyHandle hdl, const char* buf, u32 bufLen, FastlyWriteEnd mode, u32& numWritten ); // Frees the body on the host. For streaming bodies, terminates the stream, which signals to the client that the stream complete. F_DEF( fastly_http_body, close ) ( FastlyHttpBodyHandle hdl ); // For streaming bodies, terminates the stream, that the stream is not complete. Not supported in Viceroy. F_DEF( fastly_http_body, abandon ) ( FastlyHttpBodyHandle hdl ); F_DEF( fastly_http_body, trailer_append ) ( FastlyHttpBodyHandle hdl, const char* name, u32 nameLen, const char* value, u32 valLen ); F_DEF( fastly_http_body, trailer_value_get ) ( FastlyHttpBodyHandle hdl, const char* name, u32 nameLen, char* buf, u32 bufLen, u32& numRead ); F_DEF( fastly_http_body, trailer_names_get ) ( FastlyHttpBodyHandle hdl, char* buf, u32 bufLen, u32 cursor, u32& endingCursor, u32& numRead ); F_DEF( fastly_http_body, trailer_values_get ) ( FastlyHttpBodyHandle hdl, const char* name, u32 nameLen, char* buf, u32 bufLen, u32 cursor, u32& endingCursor, u32& numRead ); F_DEF( fastly_http_body, known_length ) ( FastlyHttpBodyHandle hdl, u64& length ); // ---------------- Misc ------------------------------------------------------------------------------- F_DEF( fastly_log, endpoint_get ) ( const char* name, u32 nameLen, FastlyLogHandle& hdl ); F_DEF( fastly_log, write ) ( FastlyLogHandle hdl, const char* str, u32 len, u32& numWritten ); F_DEF( fastly_geo, lookup ) ( const u8* ipAddr, u32 addrLen, const char* buf, u32 bufLen, u32& numRead ); F_DEF( fastly_device_detection, lookup ) ( const char* userAgentString, const char* buf, u32 bufLen, u32& numRead ); // ---------------- HttpResp ------------------------------------------------------------------------------- F_DEF( fastly_http_resp, new ) ( FastlyHttpRespHandle& hdl ); F_DEF( fastly_http_resp, header_names_get ) ( FastlyHttpRespHandle hdl, char* buf, u32 bufLen, u32 cursor, u32& endingCursor, u32& bytesWritten ); F_DEF( fastly_http_resp, header_value_get ) ( FastlyHttpRespHandle hdl, const char* name, u32 nameLen, char* buf, u32 bufLen, u32& readLen ); F_DEF( fastly_http_resp, header_values_get ) ( FastlyHttpRespHandle hdl, const char* name, u32 nameLen, char* buf, u32 bufLen, u32 cursor, u32& endingCursor, u32& numRead ); F_DEF( fastly_http_resp, header_values_set ) ( FastlyHttpRespHandle hdl, const char* name, u32 nameLen, const char* values, u32 valLen ); // values is an array of strings separated by \0 F_DEF( fastly_http_resp, header_insert ) ( FastlyHttpRespHandle hdl, const char* name, u32 nameLen, const char* value, u32 valLen ); F_DEF( fastly_http_resp, header_append ) ( FastlyHttpRespHandle hdl, const char* name, u32 nameLen, const char* value, u32 valLen ); F_DEF( fastly_http_resp, header_remove ) ( FastlyHttpRespHandle hdl, const char* name, u32 nameLen ); F_DEF( fastly_http_resp, version_get ) ( FastlyHttpRespHandle hdl, FastlyHttpVersion& httpVersion ); F_DEF( fastly_http_resp, version_set ) ( FastlyHttpRespHandle hdl, FastlyHttpVersion httpVersion ); F_DEF( fastly_http_resp, status_get ) ( FastlyHttpRespHandle hdl, FastlyHttpStatus& status ); F_DEF( fastly_http_resp, status_set ) ( FastlyHttpRespHandle hdl, FastlyHttpStatus status ); F_DEF( fastly_http_resp, send_downstream ) ( FastlyHttpRespHandle hdl, FastlyHttpBodyHandle bhdl, u32 streaming ); F_DEF( fastly_http_resp, close ) ( FastlyHttpRespHandle hdl ); F_DEF( fastly_http_resp, framing_headers_mode_set ) ( FastlyHttpRespHandle hdl, FastlyFramingMode mode ); F_DEF( fastly_http_resp, http_keepalive_mode_set ) ( FastlyHttpRespHandle hdl, FastlyKeepAliveMode mode ); // get the destination IP used for this request. The buffer for the IP address must be 16 bytes. This will return // the number of bytes written to the buffer: 4 for IPv4 addresses, and 16 for IPv6. F_DEF( fastly_http_resp, get_addr_dest_ip ) ( FastlyHttpRespHandle hdl, u8* octetsBuf, u32& bytesRead ); F_DEF( fastly_http_resp, get_addr_dest_port ) ( FastlyHttpRespHandle hdl, u32& port ); // ---------------- HttpReq ------------------------------------------------------------------------------- F_DEF( fastly_http_req, new ) ( FastlyHttpReqHandle& hdl ); F_DEF( fastly_http_req, body_downstream_get ) ( FastlyHttpReqHandle& hdl, FastlyHttpBodyHandle& bhdl ); F_DEF( fastly_http_req, cache_override_set ) ( FastlyHttpReqHandle hdl, FastlyCacheOverrideTag tag, u32 ttl, u32 stale_revalidate ); F_DEF( fastly_http_req, cache_override_v2_set ) ( FastlyHttpReqHandle hdl, FastlyCacheOverrideTag tag, u32 ttl, u32 stale_revalidate, const u8* sk, u32 skLen ); F_DEF( fastly_http_req, downstream_client_ip_addr ) ( u8* octetsBuf, u32& bytesRead ); F_DEF( fastly_http_req, downstream_server_ip_addr ) ( u8* octetsBuf, u32& bytesRead ); F_DEF( fastly_http_req, downstream_client_h2_fingerprint ) ( FastlyHttpReqHandle hdl, u8* buf, u32 bufLen, u32& bytesRead ); F_DEF( fastly_http_req, downstream_client_request_id ) ( FastlyHttpReqHandle hdl, u8* buf, u32 bufLen, u32& bytesRead ); F_DEF( fastly_http_req, downstream_client_oh_fingerprint ) ( FastlyHttpReqHandle hdl, u8* buf, u32 bufLen, u32& bytesRead ); F_DEF( fastly_http_req, downstream_tls_cipher_openssl_name ) ( FastlyHttpReqHandle hdl, u8* buf, u32 bufLen, u32& bytesRead ); F_DEF( fastly_http_req, downstream_tls_protocol ) ( FastlyHttpReqHandle hdl, u8* buf, u32 bufLen, u32& bytesRead ); F_DEF( fastly_http_req, downstream_tls_client_hello ) ( FastlyHttpReqHandle hdl, u8* buf, u32 bufLen, u32& bytesRead ); F_DEF( fastly_http_req, downstream_tls_raw_client_certificate ) ( FastlyHttpReqHandle hdl, u8* buf, u32 bufLen, u32& bytesRead ); F_DEF( fastly_http_req, downstream_tls_client_cert_verify_result ) ( FastlyHttpReqHandle hdl, FastlyCertVerifyResult& result ); F_DEF( fastly_http_req, downstream_tls_ja3_md5 ) ( FastlyHttpReqHandle hdl, u8* buf, u32 bufLen, u32& bytesRead ); F_DEF( fastly_http_req, downstream_tls_ja4 ) ( FastlyHttpReqHandle hdl, u8* buf, u32 bufLen, u32& bytesRead ); F_DEF( fastly_http_req, downstream_compliance_region ) ( FastlyHttpReqHandle hdl, u8* buf, u32 bufLen, u32& bytesRead ); F_DEF( fastly_http_req, header_names_get ) ( FastlyHttpReqHandle hdl, char* buf, u32 bufLen, u32 cursor, u32& endingCursor, u32& bytesWritten ); F_DEF( fastly_http_req, original_header_names_get ) ( FastlyHttpReqHandle hdl, char* buf, u32 bufLen, u32 cursor, u32& endingCursor, u32& bytesWritten ); F_DEF( fastly_http_req, original_header_count ) ( FastlyHttpReqHandle hdl, u32& headerCount ); F_DEF( fastly_http_req, header_value_get ) ( FastlyHttpReqHandle hdl, const char* name, u32 nameLen, char* buf, u32 bufLen, u32& readLen ); F_DEF( fastly_http_req, header_values_get ) ( FastlyHttpReqHandle hdl, const char* name, u32 nameLen, char* buf, u32 bufLen, u32 cursor, u32& endingCursor, u32& numRead ); F_DEF( fastly_http_req, header_values_set ) ( FastlyHttpReqHandle hdl, const char* name, u32 nameLen, const char* values, u32 valLen ); // values is an array of strings separated by \0 F_DEF( fastly_http_req, header_insert ) ( FastlyHttpReqHandle hdl, const char* name, u32 nameLen, const char* value, u32 valLen ); F_DEF( fastly_http_req, header_append ) ( FastlyHttpReqHandle hdl, const char* name, u32 nameLen, const char* value, u32 valLen ); F_DEF( fastly_http_req, header_remove ) ( FastlyHttpReqHandle hdl, const char* name, u32 nameLen ); F_DEF( fastly_http_req, method_get ) ( FastlyHttpReqHandle hdl, char* buf, u32 bufLen, u32& numRead ); F_DEF( fastly_http_req, method_set ) ( FastlyHttpReqHandle hdl, const char* buf, u32 bufLen ); F_DEF( fastly_http_req, uri_get ) ( FastlyHttpReqHandle hdl, char* buf, u32 bufLen, u32& numRead ); F_DEF( fastly_http_req, uri_set ) ( FastlyHttpReqHandle hdl, const char* buf, u32 bufLen ); F_DEF( fastly_http_req, version_get ) ( FastlyHttpReqHandle hdl, FastlyHttpVersion& httpVersion ); F_DEF( fastly_http_req, version_set ) ( FastlyHttpReqHandle hdl, FastlyHttpVersion httpVersion ); F_DEF( fastly_http_req, close ) ( FastlyHttpReqHandle hdl ); /* F_DEF( fastly_http_req, send ) ( FastlyHttpReqHandle hdl ); F_DEF( fastly_http_req, send_v2 ) ( FastlyHttpReqHandle hdl ); F_DEF( fastly_http_req, send_async ) ( FastlyHttpReqHandle hdl ); F_DEF( fastly_http_req, send_async_streaming ) ( FastlyHttpReqHandle hdl ); F_DEF( fastly_http_req, pending_req_poll_v2 ) ( FastlyHttpReqHandle hdl ); F_DEF( fastly_http_req, pending_req_wait_v2 ) ( FastlyHttpReqHandle hdl ); F_DEF( fastly_http_req, pending_req_select ) ( FastlyHttpReqHandle hdl ); F_DEF( fastly_http_req, pending_req_select_v2 ) ( FastlyHttpReqHandle hdl ); F_DEF( fastly_http_req, fastly_key_is_valid ) ( FastlyHttpReqHandle hdl ); F_DEF( fastly_http_req, auto_decompress_response_set ) ( FastlyHttpReqHandle hdl ); F_DEF( fastly_http_req, upgrade_websocket ) ( FastlyHttpReqHandle hdl ); F_DEF( fastly_http_req, redirect_to_websocket_proxy ) ( FastlyHttpReqHandle hdl ); F_DEF( fastly_http_req, redirect_to_websocket_proxy_v2 ) ( FastlyHttpReqHandle hdl ); F_DEF( fastly_http_req, redirect_to_grip_proxy ) ( FastlyHttpReqHandle hdl ); F_DEF( fastly_http_req, redirect_to_grip_proxy_v2 ) ( FastlyHttpReqHandle hdl ); F_DEF( fastly_http_req, framing_headers_mode_set ) ( FastlyHttpReqHandle hdl ); F_DEF( fastly_http_req, register_dynamic_backend ) ( FastlyHttpReqHandle hdl ); */ /* F_DEF( fastly_backend, exists ) ( ); F_DEF( fastly_backend, is_healthy ) ( ); F_DEF( fastly_backend, is_dynamic ) ( ); F_DEF( fastly_backend, get_host ) ( ); F_DEF( fastly_backend, get_override_host ) ( ); F_DEF( fastly_backend, get_port ) ( ); F_DEF( fastly_backend, get_connect_timeout_ms ) ( ); F_DEF( fastly_backend, get_first_byte_timeout_ms ) ( ); F_DEF( fastly_backend, get_between_bytes_timeout_ms ) ( ); F_DEF( fastly_backend, is_ssl ) ( ); F_DEF( fastly_backend, get_ssl_min_version ) ( ); F_DEF( fastly_backend, get_ssl_max_version ) ( ); F_DEF( fastly_cache, lookup ) ( ); F_DEF( fastly_cache, insert ) ( ); F_DEF( fastly_cache, transaction_lookup ) ( ); F_DEF( fastly_cache, transaction_insert ) ( ); F_DEF( fastly_cache, transaction_insert_and_stream_back ) ( ); F_DEF( fastly_cache, transaction_update ) ( ); F_DEF( fastly_cache, transaction_cancel ) ( ); F_DEF( fastly_cache, close ) ( ); F_DEF( fastly_cache, get_state ) ( ); F_DEF( fastly_cache, get_user_metadata ) ( ); F_DEF( fastly_cache, get_body ) ( ); F_DEF( fastly_cache, get_length ) ( ); F_DEF( fastly_cache, get_max_age_ns ) ( ); F_DEF( fastly_cache, get_stale_while_revalidate_ns ) ( ); F_DEF( fastly_cache, get_age_ns ) ( ); F_DEF( fastly_cache, get_hits ) ( ); F_DEF( fastly_uap, [resource_drop]user_agent ) ( ); F_DEF( fastly_async_io, select ) ( ); F_DEF( fastly_async_io, is_ready ) ( ); F_DEF( fastly_dictionary open ) ( ); F_DEF( fastly_dictionary get ) ( ); F_DEF( fastly_purge, purge_surrogate_key ) ( ); F_DEF( fastly_kv_store, open ) ( ); F_DEF( fastly_kv_store, lookup ) ( ); F_DEF( fastly_kv_store, lookup_async ) ( ); F_DEF( fastly_kv_store, pending_lookup_wait ) ( ); F_DEF( fastly_kv_store, insert ) ( ); F_DEF( fastly_kv_store, insert_async ) ( ); F_DEF( fastly_kv_store, pending_insert_wait ) ( ); F_DEF( fastly_kv_store, delete_async ) ( ); F_DEF( fastly_kv_store, pending_delete_wait ) ( ); F_DEF( fastly_uap, [method]user_agent.family ) ( ); F_DEF( fastly_uap, [method]user_agent.major ) ( ); F_DEF( fastly_uap, [method]user_agent.minor ) ( ); F_DEF( fastly_uap, [method]user_agent.patch ) ( ); F_DEF( fastly_uap, parse ) ( ); F_DEF( fastly_erl, check_rate ) ( ); F_DEF( fastly_erl, ratecounter_increment ) ( ); F_DEF( fastly_erl, ratecounter_lookup_rate ) ( ); F_DEF( fastly_erl, ratecounter_lookup_count ) ( ); F_DEF( fastly_erl, penaltybox_add ) ( ); F_DEF( fastly_erl, penaltybox_has ) ( ); F_DEF( fastly_config_store, open ) ( ); F_DEF( fastly_config_store, get ) ( ); F_DEF( fastly_secret_store, open ) ( ); F_DEF( fastly_secret_store, get ) ( ); F_DEF( fastly_secret_store, from_bytes ) ( ); F_DEF( fastly_secret_store, plaintext ) ( ); */ }
You can see that I didn’t define every function in the ABI, but once you see the pattern from an ABI expression into a header function, building the other ones should be fairly straightforward.
Building and Launching a Service
First you just build at WASM binary that calls these functions. If the function signatures are exactly correct the binary will run in Fastly’s service. All you have to do now is upload the binary.
The easiest way is using the fastly CLI. You install it like this:
# Download the appropriate .deb file for your platform from # https://github.com/fastly/cli/releases wget https://github.com/fastly/cli/releases/download/v10.17.1/fastly_10.17.1_linux_amd64.deb # Install using apt sudo apt update sudo apt install -y "./fastly_10.17.1_linux_amd64.deb"
Create a gzip file that contains your binary and a fastly .toml file. The .toml file is in the root directory, and the wasm binary is in the /bin directory.
You can text your service locally with the following command:
fastly compute serve --file=my_service.wasm
This command runs a small local web server on port 7676 with your service on it, so you can do some basic bench testing. To deploy it live you run the the following command:
fastly compute deploy --package=my_service.tar.gz
A real production deployment environment is a little more complex, because you want to have pre-release and release versions of services, the ability to roll back etc. But for just getting a service up and running this command should work.