CelerData Quarterly Update: Automatic Materialized Views and More

We are excited to share the latest round of major updates to CelerData Cloud, including features aimed at improving efficiency, usability, security, and accessibility for your customer-facing analytics workloads. In this article, we will introduce 5 major features that build on top of StarRocks to elevate how you work with data lakes, reduce operational complexity, and ensure enterprise-grade disaster recovery.

Unity Catalog Integration (Public Preview)

Unity Catalog, a newly open-sourced catalog service from Databricks, allows for open and interoperable data governance across data and AI workloads. By supporting multiple engines on a single data copy, it simplifies data management and enhances flexibility.

CelerData Cloud’s Unity Catalog integration optimizes for high-concurrency, low-latency workloads, offering seamless connections to catalog services without complex metadata discovery. Together with native Delta Lake support, it enables faster, more efficient customer-facing workloads for teams using Databricks or the open-source Unity Catalog.

Automatic Materialized Views (Public Preview)

With powerful query rewrite capabilities, StarRocks’ materialized views (MV) are designed to accelerate your most critical workloads on demand. Previously, fully leveraging materialized views required certain knowledge:

• Understanding query patterns and data distribution to identify optimal opportunities for materialized views and create them with optimal configurations such as partitioning.

• Familiarity with the internals of materialized view rewrites in StarRocks to create views that can be efficiently rewritten.

• Designing generalized materialized views that accelerate a wide range of queries, helping to manage and control costs effectively.

For many teams, these technical demands make it difficult to take full advantage of materialized views’ performance potential. Automatic Materialized View in CelerData Cloud removes this complexity by recommending materialized views based on your query pattern and data distribution. Through this new feature, CelerData Cloud optimizes query performance without the need for specialized technical knowledge.

• Ease of Use: Empowers teams to harness the benefits of materialized views effortlessly, bypassing the steep learning curve of manual setup.

• Accelerating More Queries: Enjoy customer-facing performance gains automatically, with no manual intervention required.

• Lowered TCO: Automate query optimization, reducing the time, effort, and resources needed for MV management.

Failover Group for Disaster Recovery (Public Preview)

Ensuring high availability and minimizing downtime during unexpected failures is essential. Without dedicated features for disaster recovery, teams often struggle to meet critical Recovery Time Objective (RTO) and Recovery Point Objective (RPO) requirements. Achieving effective recovery can become a complex and resource-intensive process, often involving manual deployment and intricate cross-cluster migration setups.

CelerData Cloud now introduces the Failover Group feature to simplify disaster recovery and maintain availability.

• Enhanced Recovery Time: Streamlined data ingestion job syncing enables rapid data recovery, ensuring swift service resumption after failures.

• Minimized Data Loss: Optimizes data integrity by reducing potential data loss, allowing teams to meet strict RPO targets.

• Simplified Management: Eliminates the need for complex cross-cluster migration tools, making disaster recovery more straightforward and less resource-intensive.

• Reliable High Availability: Automates failover processes, maintaining continuous availability and providing a resilient foundation for data operations.

Transparent Data Encryption (Public Preview)

Data security is non-negotiable. With increasing regulatory standards, enterprises are required to encrypt data to meet compliance requirements and internal security policies. CelerData Cloud enhances StarRocks with Transparent Data Encryption (TDE). This feature supports the use of custom encryption keys, whether from Key Management Services (KMS) or other vault solutions like HashiCorp Vault.

This encryption capability offers several key benefits for CelerData Cloud users:

• Enhanced Efficiency: Encrypts data at rest with less than 10% performance overhead, ensuring strong security without impacting system efficiency.

• Easy to Use: Automates key rotation, reducing manual administrative tasks and enhancing security by keeping encryption keys current.

• Seamless Integration: Fully compatible with existing workflows, requiring no changes to SQL for smooth, immediate integration.

Compute Autoscaling (Public Preview)

Making sure your compute resources can dynamically scale according to the workload is essential to maintaining performance and controlling costs. CelerData Cloud’s Compute Autoscaling allows users to define custom autoscaling policies for each warehouse, enabling adaptive adjustments to the number of Compute Nodes for each compute warehouse based on real-time CPU utilization. This feature delivers stable, predictable performance while optimizing costs by scaling resources in or out according to set thresholds.

With Compute Autoscaling, you can:

• Adapt to Workload Changes: Automatically scale up or down based on real-time CPU utilization, maintaining consistent performance across variable workloads.

• Optimize Costs: Use only the resources you need, reducing costs by scaling in during low usage periods.

• Flexible Policy Management: Define custom scaling policies for each warehouse, giving you full control over minimum and maximum Compute Node counts, scaling thresholds, and timing.

This feature allows CelerData Cloud users to achieve even more efficient, cost-effective operations without manual resource management, ensuring resources always match demand.

Get Started Now

From Unity Catalog integration for seamless data governance to automatic materialized views, autoscaling, TDE, and robust disaster recovery with the Failover Group, these enhancements build on StarRocks to simplify operations and bolster resilience.

See these exclusive CelerData Cloud features firsthand. Register now for a live webinar to dive deeper into these features with demos.