Unreal Cloud DDC Single Region¶

The Unreal Cloud DDC Single Region is a comprehensive solution that leverages several AWS services to create a robust and efficient data caching system. It uses a well-designed Virtual Private Cloud (VPC) to ensure network isolation and security. The solution employs an Amazon Elastic Kubernetes Service (EKS) Cluster with Node Groups to manage and orchestrate containerized applications.

At the heart of the system is an instance of ScyllaDB, a high-performance NoSQL database, running on specially optimized Amazon EC2 instances. The Unreal Cloud Derived Data Cache Container is managed by Helm, a package manager for Kubernetes, and uses Amazon S3 for durable storage.

Predeployment - Set Up Github Content Repository Credentials¶

The Unreal Cloud DDC Intra Cluster module utilizes a pull through cache to access the Unreal Cloud DDC image. This requires a secret in Secrets Manager. The secret needs to be prefixed with ecr-pullthroughcache/. Additionally, the secret is required to be in the following format:

{
  "username":"GITHUB-USER-NAME-PLACEHOLDER",
  "access-token":"GITHUB-ACCESS-TOKEN-PLACEHOLDER"
}

Deployment¶

Once you've completed the prerequisites and set your variables, you can deploy the solution by running:

terraform apply

The deployment can take close to 30 minutes. Creating the EKS Node Groups and EKS Cluster take around 20 minutes to fully deploy.

Postdeployment¶

The sample deploys a Route53 dns record that you can use to access your Unreal DDC cluster. This record points to an NLB which may take more time to become fully available when the deployment is complete. You can view the provisioning status of this NLB on the EC2 load balncing screen.

The Unreal Cloud DDC module creates a Service Account and valid bearer token for testing. This bearer token is stored in AWS Secrets Manager. The ARN of this secret is provided as a Terraform output ("unreal_cloud_ddc_bearer_token_arn") on the console following deployment. To fetch the bearer token you can use the aws CLI:

aws secretsmanager get-secret-value --secret-id <"unreal_cloud_ddc_bearer_token_arn">

To validate you can put an object you can run:

curl http://<unreal_ddc_url>/api/v1/refs/ddc/default/00000000000000000000000000000000000000aa -X PUT --data 'test' -H 'content-type: application/octet-stream' -H 'X-Jupiter-IoHash: 4878CA0425C739FA427F7EDA20FE845F6B2E46BA' -i -H 'Authorization: ServiceAccount <secret-manager-token>'

After running this you should get a response that looks as the following:

HTTP/1.1 200 OK
Server: nginx
Date: Wed, 29 Jan 2025 19:15:05 GMT
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Server-Timing: blob.put.FileSystemStore;dur=0.1451;desc="PUT to store: 'FileSystemStore'",blob.put.AmazonS3Store;dur=267.0449;desc="PUT to store: 'AmazonS3Store'",blob.get-metadata.FileSystemStore;dur=0.0406;desc="Blob GET Metadata from: 'FileSystemStore'",ref.finalize;dur=7.1407;desc="Finalizing the ref",ref.put;dur=25.2064;desc="Inserting ref"

{"needs":[]}%

You can then access the same chunk with the following command:

curl http://<unreal_ddc_url>/api/v1/refs/ddc/default/00000000000000000000000000000000000000aa.json -i -H 'Authorization: ServiceAccount <unreal-cloud-ddc-bearer-token>'

The response should look like the following:

HTTP/1.1 200 OK
Server: nginx
Date: Wed, 29 Jan 2025 19:16:46 GMT
Content-Type: application/json
Content-Length: 66
Connection: keep-alive
X-Jupiter-IoHash: 7D873DCC262F62FBAA871FE61B2B52D715A1171E
X-Jupiter-LastAccess: 01/29/2025 19:16:46
Server-Timing: ref.get;dur=0.0299;desc="Fetching Ref from DB"

{"RawHash":"4878ca0425c739fa427f7eda20fe845f6b2e46ba","RawSize":4}%

For a more comprehensive test of your deployment, we recommend using the bench marking tools. To do so we used a x2idn.32xlarge as it matched Epic's benchmarking instance to test their configuration.

With the benchmarking tools we ran the following command after compiling the docker image:

docker run --network host jupiter_benchmark --seed --seed-remote --host http://<unreal_ddc_url> --namespace ddc \
--header="Authorization: ServiceAccount <unreal-cloud-ddc-bearer-token>" all

Just a note here, you will have to specify the namespace to be DDC as the token only has access to that namespace.

It is recommended that if you are using this in a production capacity you change the authentication mode from Service Account to Bearer and use an IDP to authenticate and TLS termination.

This sample also deploys a ScyllaDB monitoring stack, enabling real-time insights into the status and performance of your ScyllaDB nodes. The monitoring stack includes Prometheus for metrics collection, Alertmanager for handling alerts, and Grafana for visualization. You can access the Grafana dashboard by using the "monitoring_url" provided in the sample outputs. To learn more about the ScyllaDB monitoring stack, refer to the ScyllaDB Monitoring Stack Documentation.

Requirements¶

Name	Version
terraform	>= 1.10.3
aws	>= 5.89.0
awscc	>= 1.26.0
helm	>= 2.9.0, < 3.0.0
http	>= 3.4.5
kubernetes	>= 2.24.0
random	3.7.2

Providers¶

Name	Version
aws	>= 5.89.0
awscc	>= 1.26.0
http	>= 3.4.5

Modules¶

Name	Source	Version
unreal_cloud_ddc_infra	../../modules/unreal/unreal-cloud-ddc/unreal-cloud-ddc-infra	n/a
unreal_cloud_ddc_intra_cluster	../../modules/unreal/unreal-cloud-ddc/unreal-cloud-ddc-intra-cluster	n/a
unreal_cloud_ddc_vpc	./vpc	n/a

Resources¶

Name	Type
aws_acm_certificate.scylla_monitoring	resource
aws_acm_certificate_validation.scylla_monitoring	resource
aws_route53_record.scylla_monitoring	resource
aws_route53_record.scylla_monitoring_cert	resource
aws_route53_record.unreal_cloud_ddc	resource
aws_security_group.unreal_ddc_load_balancer_access_security_group	resource
aws_vpc_security_group_egress_rule.unreal_ddc_load_balancer_egress_sg_rules	resource
aws_vpc_security_group_ingress_rule.unreal_ddc_load_balancer_http2_ingress_rule	resource
aws_vpc_security_group_ingress_rule.unreal_ddc_load_balancer_http_ingress_rule	resource
aws_vpc_security_group_ingress_rule.unreal_ddc_load_balancer_https_ingress_rule	resource
awscc_secretsmanager_secret.unreal_cloud_ddc_token	resource
aws_availability_zones.available	data source
aws_caller_identity.current	data source
aws_ecr_authorization_token.token	data source
aws_region.current	data source
aws_route53_zone.root	data source
aws_secretsmanager_secret_version.unreal_cloud_ddc_token	data source
http_http.public_ip	data source

Inputs¶

Name	Description	Type	Default	Required
github_credential_arn	Github Credential ARN	`string`	n/a	yes
route53_public_hosted_zone_name	The root domain name for the Hosted Zone where the ScyllaDB monitoring record should be created.	`string`	n/a	yes
allow_my_ip	Automatically add your IP to the security groups allowing access to the Unreal DDC and SycllaDB Monitoring load balancers	`bool`	`true`	no

Outputs¶

Name	Description
monitoring_url	n/a
unreal_cloud_ddc_bearer_token_arn	n/a
unreal_ddc_url	n/a