Building SG Bus Arrival: My Journey
Introduction
This journey started with the goal of creating a reliable application that displays bus arrival timings in Singapore, powered by data from multiple sources. The project evolved into a fascinating technical challenge as I encountered various obstacles with data integration and mapping routes.
Step 1: Getting Started with LTA Data
The foundation of the project began with the Land Transport Authority (LTA) of Singapore’s data API, which provides bus arrival timings and bus stop information. It seemed straightforward at first, but soon I realized the complexity would go beyond just fetching and displaying data.
Step 2: OpenStreetMap & The Lat/Lng Mismatch
To enhance the user experience, I wanted to display bus stops and routes on a map. I decided to use OpenStreetMap (OSM) for its rich geospatial data. However, a key challenge arose: the latitude and longitude coordinates provided by LTA did not always match perfectly with OpenStreetMap data.
Step 3: Solving Mismatched Data with Overpass API
To address the mismatch issue, I used Overpass API to query OpenStreetMap data based on LTA’s bus stop codes. This allowed me to get more accurate latitude and longitude coordinates for each bus stop, ensuring they were displayed correctly on the map. However, I encountered some missing bus stops in Overpass API. Since I believe LTA’s data is more up-to-date, I manually checked the missing stops one by one. So far, the results look good, but it will be an ongoing, iterative process to ensure the data stays accurate.
Step 4: The Missing Routes Problem
Another limitation I discovered in the LTA API was that it only provides bus stops for each service, not the actual routes. To display the routes on a map, I needed more comprehensive data, so I began exploring alternative sources.
Step 5: OpenRouteService for Route Data
I used OpenRouteService (ORS) to get route data. ORS, combined with Overpass API, provided a solution for plotting entire bus routes, allowing users to visualize the paths that buses take through Singapore. The integration of these tools allowed me to offer a more complete view of the bus routes for users.
Step 6: Overcoming the Data Mismatch
One of the key challenges was handling data mismatches between LTA, OpenStreetMap, and OpenRouteService. Despite my efforts, there were occasional inconsistencies, which required implementing workarounds to ensure the data was consistent enough for accurate route mapping.
Tech Stack
- React Framework: Next.js
- State Management: Apollo for GraphQL, Jotai for app shared state
- Database: Postgres
- ORM: Drizzle
Conclusion
This project has been a valuable learning experience, from handling the LTA API to integrating various open-source mapping tools. I’ve learned a lot about geospatial data, API integration, and problem-solving along the way. Moving forward, I’m excited to continue refining the application, improving data accuracy, and expanding its features.
References
- LTA DataMall API - Provides bus arrival timings and bus stop information in Singapore.
- OpenStreetMap - A collaborative mapping platform used for rich geospatial data.
- Overpass API - A tool to query OpenStreetMap data for specific geographic elements.
- OpenRouteService API - Provides routing services for geographic data, used for plotting bus routes.