big data - aitoearn

big data

Cloud Platforms Overwhelmed? Poizon’s Breakthrough in Building Its Own Big Data R&D and Management Platform

# Galaxy Data Development & Management Platform — Technical Overview ## Table of Contents --- **I. Background Introduction** **II. Product Functional Architecture** **III. The "Cockpit" of Data Development — Data R&D Suite** 1. System Architecture Analysis 2. Data Synchronization Technology Analysis 3. Task Migration Solutions 4. Functional Development & Migration

JavaScript

[Morning Read] Breaking Big Data Limits in the Browser

🚀 A New Approach: Large-Scale Data Apps in Pure JavaScript Build large-scale data applications entirely in JavaScript — no Python required. This talk introduces Hyparquet and HighTable, open-source libraries enabling browsers to load Apache Parquet files directly. --- 👤 Introduction Speaker: Kenny Daniel, AI-focused startup founder in Seattle. Main themes: * Performance bottlenecks in

data management

Farewell to Data Chaos: The Breakthrough Path of Dewu’s Data R&D and Management Platform

# Galaxy Data Development and Management Platform **Shanghai — 2025-10-27** The **Galaxy Data Development and Management Platform** is designed for data production teams, providing: - **Offline & real-time** data collection and synchronization - **Development & operations** tooling - **Processing & production** pipelines - **Data asset management** frameworks - **Security & compliance** controls

database pagination

How to Paginate Queries with 10 Million Records in a Database

How to Handle Big Data Pagination in Real-World Business This guide explores a classic Alibaba Cloud interview question: > How do you implement pagination queries for tens of millions of records? Beyond interviews, this is a real performance-critical challenge in production systems. We’ll walk through solutions from basic methods

big data

Save 20+ Person-Months! Huolala’s Offline Big Data Cross-Cloud Migration Final Review

1. Preface At the end of 2023, our company officially launched the Freight Offline Big Data Migration Project. After five months of intensive, cross-department collaboration, the project was successfully completed in May 2024, achieving a full-scale cross-cloud migration of offline pipelines — covering tasks, data, services, and infrastructure. Over ten departments

HDFS

vivo HDFS EC Large-Scale Implementation Practices

Table of Contents * 01. Principle of EC Codes * 02. Changes in Storage Layout * 03. Practical Application of HDFS EC Codes * 04. Summary & Outlook --- Quick Overview – Grasp the Essentials in 1 Minute Erasure Coding (EC) is a data protection technique that allows recovery from partial data loss. Introduced in