This architecture supports efficient and scalable search functionality for healthcare listings, ensuring quick and accurate responses to user queries. The system leverages Apache Kafka for real-time messaging, Elasticsearch for high-performance full-text searching, and a Django backend to handle API requests.
1. User Interaction via Frontend
- Interface (React/Next.js):
- Customers/patients interact with the application via a search interface.
- They input search queries (e.g., “Dental services near me”).
- The frontend sends these queries to the backend through REST API requests.
2. API Gateway and Backend Processing
- Elastic Load Balancer (ELB):
- Balances incoming traffic to the backend services, ensuring high availability and fault tolerance.
- Django Backend (Search Backend Service):
- The backend service processes the user queries, parses the search criteria, and interacts with Elasticsearch to fetch relevant results.
- Key responsibilities:
- Validating and handling API requests.
- Querying Elasticsearch for fast and precise search results.
3. Elasticsearch for Full-Text Search
- Elasticsearch:
- Acts as the primary search engine, storing and indexing data such as healthcare facilities, service details, locations, etc.
- Features like fuzzy search improve the system’s ability to handle typos or partial matches.
- Elasticsearch indexes the healthcare data received from the Kafka Consumer, enabling quick retrieval of search results.
4. Data Updates via Apache Kafka
- Kafka as a Message Broker:
- Kafka efficiently streams updates related to healthcare data (e.g., new facilities, modified details).
- Producers (such as the Healthcare Service) send data updates to the
hc-updatesKafka topic. - Kafka Consumer (ElasticSearch Consumer):
- The consumer service listens to updates from the
hc-updatestopic. - It processes these updates and re-indexes the affected data in Elasticsearch, ensuring the search engine always reflects the most current information.
5. Data Source (Healthcare Service and MySQL DB)
- Healthcare Service:
- Responsible for managing and updating healthcare-related data, such as facilities, services, and details.
- When data changes occur (e.g., new facility added), the service updates the MySQL Healthcare DB.
- Simultaneously, it publishes update messages to the
hc-updatesKafka topic. - MySQL Healthcare Database:
- Stores the source data for healthcare listings, serving as the central database for structured data.
The healthcare search mechanism combines technologies like Django, Elasticsearch, Kafka, and Redis to deliver fast, scalable, and accurate results. With efficient workflows for query handling and real-time data indexing, the system ensures an optimal user experience while maintaining data consistency across services.
Part 1: Building a Scalable Healthcare Booking Platform (Part 1): A System Story
Part 2: Building a Scalable Healthcare Booking Platform (Part 2): Technical Process Overview
Part 4: Building a Scalable Healthcare Booking Platform (Part 4): Role-Based Access Control