Unreal Cloud DDC Single Region¶

The Unreal Cloud DDC Single Region is a comprehensive solution that leverages several AWS services to create a robust and efficient data caching system. It uses a well-designed Virtual Private Cloud (VPC) to ensure network isolation and security. The solution employs an Amazon Elastic Kubernetes Service (EKS) Cluster with Node Groups to manage and orchestrate containerized applications.

At the heart of the system is an instance of ScyllaDB, a high-performance NoSQL database, running on specially optimized Amazon EC2 instances. The Unreal Cloud Derived Data Cache Container is managed by Helm, a package manager for Kubernetes, and uses Amazon S3 for durable storage.

Predeployment - Set Up Github Content Repository Credentials¶

The Unreal Cloud DDC Inter Cluster module utilizes a pull through cache to access the Unreal Cloud DDC image. This requires a secret in Secrets Manager. The secret needs to be prefixed with ecr-pullthroughcache/. Additionally, the secret is required to be in the following format:

{
  "username":"GITHUB-USER-NAME-PLACEHOLDER",
  "accessToken":"GITHUB-ACCESS-TOKEN-PLACEHOLDER"
}

Deployment¶

Once you've completed the prerequisites and set your variables, you can deploy the solution by running:

terraform apply

The deployment can take close to 30 minutes. Creating the EKS Node Groups and EKS Cluster take around 20 minutes to fully deploy.

Postdeployment¶

The sample deploys a Route53 dns record that you can use to access your Unreal DDC cluster. This record points to an NLB which may take more time to become fully available when the deployment is complete. You can view the provisioning status of this NLB on the EC2 load balncing screen.

The Unreal Cloud DDC module creates a Service Account and valid bearer token for testing. This bearer token is stored in AWS Secrets Manager. The ARN of this secret is provided as a Terraform output ("unreal_cloud_ddc_bearer_token_arn") on the console following deployment. To fetch the bearer token you can use the aws CLI:

aws secretsmanager get-secret-value --secret-id <"unreal_cloud_ddc_bearer_token_arn">

To validate you can put an object you can run:

curl http://<unreal_ddc_url>/api/v1/refs/ddc/default/00000000000000000000000000000000000000aa -X PUT --data 'test' -H 'content-type: application/octet-stream' -H 'X-Jupiter-IoHash: 4878CA0425C739FA427F7EDA20FE845F6B2E46BA' -i -H 'Authorization: ServiceAccount <secret-manager-token>'

After running this you should get a response that looks as the following:

HTTP/1.1 200 OK
Server: nginx
Date: Wed, 29 Jan 2025 19:15:05 GMT
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Server-Timing: blob.put.FileSystemStore;dur=0.1451;desc="PUT to store: 'FileSystemStore'",blob.put.AmazonS3Store;dur=267.0449;desc="PUT to store: 'AmazonS3Store'",blob.get-metadata.FileSystemStore;dur=0.0406;desc="Blob GET Metadata from: 'FileSystemStore'",ref.finalize;dur=7.1407;desc="Finalizing the ref",ref.put;dur=25.2064;desc="Inserting ref"

{"needs":[]}%

You can then access the same chunk with the following command:

curl http://<unreal_ddc_url>/api/v1/refs/ddc/default/00000000000000000000000000000000000000aa.json -i -H 'Authorization: ServiceAccount <unreal-cloud-ddc-bearer-token>'

The response should look like the following:

HTTP/1.1 200 OK
Server: nginx
Date: Wed, 29 Jan 2025 19:16:46 GMT
Content-Type: application/json
Content-Length: 66
Connection: keep-alive
X-Jupiter-IoHash: 7D873DCC262F62FBAA871FE61B2B52D715A1171E
X-Jupiter-LastAccess: 01/29/2025 19:16:46
Server-Timing: ref.get;dur=0.0299;desc="Fetching Ref from DB"

{"RawHash":"4878ca0425c739fa427f7eda20fe845f6b2e46ba","RawSize":4}%

For a more comprehensive test of your deployment, we recommend using the bench marking tools. To do so we used a x2idn.32xlarge as it matched Epic's benchmarking instance to test their configuration.

With the benchmarking tools we ran the following command after compiling the docker image:

docker run --network host jupiter_benchmark --seed --seed-remote --host http://<unreal_ddc_url> --namespace ddc \
--header="Authorization: ServiceAccount <unreal-cloud-ddc-bearer-token>" all

Just a note here, you will have to specify the namespace to be DDC as the token only has access to that namespace.

It is recommended that if you are using this in a production capacity you change the authentication mode from Service Account to Bearer and use an IDP to authenticate and TLS termination.

This sample also deploys a ScyllaDB monitoring stack, enabling real-time insights into the status and performance of your ScyllaDB nodes. The monitoring stack includes Prometheus for metrics collection, Alertmanager for handling alerts, and Grafana for visualization. You can access the Grafana dashboard by using the "monitoring_url" provided in the sample outputs. To learn more about the ScyllaDB monitoring stack, refer to the ScyllaDB Monitoring Stack Documentation.

Requirements¶

Name	Version
terraform	>= 1.0
aws	>= 5.38
helm	>= 2.9.0
kubernetes	>= 2.24.0

Providers¶

Name	Version
aws	>= 5.38

Modules¶

Name	Source	Version
unreal_cloud_ddc_infra	../../modules/unreal/unreal-cloud-ddc-infra	n/a
unreal_cloud_ddc_intra_cluster	../../modules/unreal/unreal-cloud-ddc-intra-cluster	n/a
unreal_cloud_ddc_vpc	git::https://github.com/terraform-aws-modules/terraform-aws-vpc.git	25322b6b6be69db6cca7f167d7b0e5327156a595

Resources¶

Name	Type
aws_region.current	data source

Inputs¶

Name	Description	Type	Default	Required
caller_ip	IPs that will be allow listed to access cluster over internet	`list(string)`	`[]`	no
ghcr_password	GHCR password	`string`	n/a	yes
ghcr_username	GHCR username	`string`	n/a	yes
jwt_audience	JWT Audience	`string`	n/a	yes
jwt_authority	JWT Authority	`string`	n/a	yes
okta_auth_server_id	Okta Auth Server ID	`string`	n/a	yes
okta_domain	Okta Domain	`string`	n/a	yes
profile	AWS Profile name	`string`	`"default"`	no
region	AWS Region	`string`	`"us-west-2"`	no

Outputs¶

No outputs.