Ministry of Transportation and Communications
Customer Background
Ministry of Transportation and Communications (MOTC) is grappling with massive amounts of data from various platforms, such as eTag with 300 million records per month, ticketing with 100 million per month, buses with 30 billion per year, etc. Roughly, they accumulate about 100 billion records weighing around 2000GB annually.
Challenges
- Poor Speed and Performance:
The original data providers frequently update or delete the raw data, requiring significant time to refresh databases. Relational databases are slow, and NoSQL databases, while fast, are difficult to maintain for structured data and aren’t suitable for direct analysis. The immense scale of the data (hundreds of billions of records each year) makes integration and processing time-consuming, preventing IT personnel from providing reliable, real-time data for business users. - Inability to Conduct Timely and Effective Analysis:
For different analytical topics each year, business users must predefine scenarios, which then require IT staff to re-acquire and develop the data. This process involves multiple rounds of communication, takes a long time, and the results may not meet user requirements.
The volume of data slows down computations. To improve the speed and effectiveness of statistical analysis, dimensions and outcomes must be pre-calculated, sacrificing the depth and flexibility of data analysis.
Customer Requirements
- Consolidate diverse data from various platforms and systems into one uniform table
- Capable of real-time, complex, and intensive processing of massive data volumes.
- Real-time access to live data, with the flexibility to explore and identify the root causes affecting traffic accidents and public transportation usage rates (Know-Why).
- Analyze electronic ticket transaction records, rail and road ticket sales/booking records, travel habit records, mobile device telecom data, and public bicycle rental records. This will help to understand public traveling patterns and behaviors, enabling real-time provision of transport services.
- Clear understanding of the supply and demand in transportation as well as travel behavior preferences to offer better customer experiences and improve service quality.
- What-If Analysis capabilities, such as providing average travel speeds, vehicle counts, and load capacity to estimate travel times and delays. Use the analysis results as references for traffic management and congestion alleviation.
- Utilize accurate data and analysis results to support decision-making in transportation services, aiming for operational optimization.
Solution
Results
Image Source: Big Object