Project Zefix Notifier
Zefix Notifier
Web
Interface
REST Server
Mysql
Email,
Callback
Reporting,
dashboarding
Admin.ch
HIVE
DATA INGESTION
Zefix
• Zefix (Zentraler Firmenindex) is the Central Business
Names Index for Switzerland.
• http://www.e-service.admin.ch/ws-zefix1.6/ZefixService?wsd
– (password protected)
• Sample of raw data provided.
Zefix Data Ingestion
Admin.ch
HDFS
DATA LOADING
From HDFS to Hive
HDFS
HIVE
• Hive table
• Partition?
• Buckets?
INGESTION AUTOMATION
Automatic data ingestion
Admin.ch
HIVE
1. Crontab
2. Oozie
USER INTERFACE
Web Interface
Web Server
Web
Interface
REST
Server
Mysql
NIGHTLY PROCESSING
Processing Workflow
Join
Load
Mysql
Filter
HIVE
NOTIFICATION
Notification
Every
Match
Email
Http callback (optional)
Let’s hack!
Fork & clone https://github.com/daplab/zefix-notifier
And let’s get started!