YAVA.
Data Management Platform.
Case Studies.

Data Warehouse Offloading

Data Warehouse implemantation developed on a big data platform

Data Lake and CDR Mediation

Improve end-to-end Business Intelligence architecture

Weather Modelling

Observed using our Weather FMCW radar

Sensor Fusion

Reduce uncertainty of sensor measurement

Operation Intelligent

Presenting customer behaviour with operation intelligent

Media Analytics

Understand media and social media to make better decisions.

Face Recognition

Biometric pattern recognition is a secure alternative to protect privacy.

Multi Target Tracking in Marine Surveillance System

Real-time multi target detection and tracking.

Data Warehouse Offloading

One of our clients, a mass transit electric train service provider is committed to improving their service quality. One key aspect of quality improvement is to embrace the data-driven operation and management. To have the strong data driven culture, they need to establish and maintain a reliable and integrated data management system. One manifestation of these measures is the implementation of data warehouse, which is developed on a big data platform.

Big data is a term used to describe large and complex data processing and data management, in which the conventional system is unable to cope with. It is largely based on an open platform with linear scalability, avoiding vendor lock-in, and giving a broader range choices and the more cost effective alternatives.

Yava and HGrid247 are chosen as the platform and tool respectively for these advantages:

1. Scalable Architecture.

Big Data Architecture is based on Hadoop technology, which is very scalable in nature. We can start with small cluster and add more nodes as the requirement increases.
The implementation can also be done gradually, by providing a training, software trial, a simple project and then implement the management and analysis of data to big data environments gradually including spatial side.

2. Longer data retention.

The advantage of this capability is to store and manage data in a structured, systematic and compressed format, so it can keep source and output data with longer retention period.

3. Best Performance with minimum cost

By having a structured, systematic and compressed data format, reports and data trace can be displayed more quickly and concisely, compared to one of regular licensed-based applications, which tend to be slow when you need to access it, and cost you periodically.

4. Long Experience in Implementation and Total Supports

Yava is built on Apache Hadoop and its supporting environments, with wide community support and adopters. Labs247’s strong experience and support in Big Data and Hadoop combined with open-source community ensures successful Yava implementation and total support of the delivered solutions.

Existing challenges to deal with:

  1. Heavy load on the data warehouse
  2. Unscalable architecture
  3. Unreliable data source from mediation
  4. Unable to provide up-to-date report
  5. High cost of hardware acquisition and maintenance

Methods to address the above points:

  1. Use commodity hardware
  2. Move ETL processes into Hadoop
  3. Keep raw & detail data

Technology Used

YAVA, HGrid247, Sqoop, Hive, HBase, Impala, Phoenix, and PHP.

Operation Intelligent

Our customer, as one of the biggest cellular providers in Indonesia, is on its effort to improve its existing end-to-end process in Operation Intelligent like data processing, capacity, quality and reporting when presenting customer behaviour by addressing these points:

  • The existing ETL process is inefficient and not capable to cover all of site regions. It only processes several machine data sources from each site. Nevertheless, we need to improve the ability, capacity and quality of the ETL for processing large volume of data in multiple types of data sources and in more frequent batches.
  • ETL processing and reporting are depending too much on file collection process. Data incompleteness and other problems in data collection will impact ETL and reporting.

Benefit

Operational Intelligent insights enable any business to:

  • Provide faster report to identify unusual issues in customer behaviour
  • Allow faster resolving unusual behaviour in CDR transaction
  • Enable faster management of more data sources with the same/increasing infrastructure capacity

YAVA for HGRID facilitates with these technology advantages:

  • More data type available
  • No dependency to Hadoop Distribution
  • Easy to use in operational and monitoring
  • Drag & drop interface
  • Provide data analysis to support decision making
  • Process large scale data faster
  • More adaptable with user requirement

Use Case

  • Collection
  • CDR Inquiry
  • Operations

Technology Used

Hadoop, HDFS, Yarn, HGrid247, Ambari, 3rd party data visualization tools

Data Lake and CDR Mediation

We help our customer, one of the biggest Cellular Operators in Indonesia improve their end-to-end Business Intelligence architecture as well as data quality of reporting to address some pain points:

  • Existing ETL process is inefficient in processing large volume of data in more frequent batches. In addition to it’s inability to process unstructured data, the licenses are expensive.
  • Reporting is depending too much on Mediation System and any problems occurring in Mediation will impact ETL, Datawarehouse and Enterprise Reporting. Mediation is a hop that slows down the overall process.

Benefit

For Business

  • Provide faster report to adapt the market
  • Allow faster customer service
  • Cost efficiency

Technically

  • More data type available
  • Scalable platform to grow the business
  • Real-time access to data
  • Lower cost of HW, SW and resources with more data
  • No dependency to Hadoop Distribution
  • More adaptable with user requirement

Use Case

  • Data Lake
  • CDR Inquiry
  • Mediation
  • Legacy Offload

Technology Used

Hadoop, HBase, Hive, HGrid247, Impala, ASN.1

Media Analytics

Media and social media have become very important parts in our everyday life. They do not only influence people’s decision, but also reflect how people think and feel about certain things. Therefore, it becomes imperative for business and decision makers to understand media and social media and what they convey, to make better decisions.

Benefit

Media and social insights enable any business to :

  • Know where to reach new and relevant customer
  • Understand what message will most effectively engage them
  • Track specific campaign and customer interaction to know what works
  • Categorize and measure customer interaction to learn about brand perception, exposure method, and future opportunities
  • Identify key areas of success to quantify ROI through changes in intent to purchase and competitor benchmarking

YAVA for Media Analysis facilitates with these advantages :

  • Easy to use in operational and monitoring
  • Drag and drop interface
  • Build topology faster
  • Complete
  • Secure
  • Enable various input (Twitter, FB, Forum, Blogs, Online News, etc)
  • Provide data analysis to support decision makers
  • Process large-scale data faster

Use Case

  • Competition Analysis
  • Brand Exposure Analysis
  • Campaign Development
  • Customer Insight Development
  • Marketing Strategy Development
  • Risk Mitigation

Technology Used

Apache Nutch, Apache Solr, Apache Storm, Apache Spark, Ambari, HGrid247, HAnalytics

Face Recognition

Wide availability of powerful and low-cost computing systems has created an enormous interest in automatic processing of digital images in a variety of applications, including biometric authentication. Biometric pattern recognition is a secure alternative to protect our privacy, in the use of replacing regular password. One of the most-known biometric pattern recognition is face recognition. Face recognition is a task that humans perform routinely and effortlessly in our daily lives. The advantages of using face recognition among other biometric recognition is: face can be captured at distance.

Existing Challenges to Deal with :

  • Low accuration on distance recognition
  • Bad face detection on low-light condition
  • Heavy matching process
  • A big database need to deal with large-deployment
  • Unscalable architecture

Methods to Address the Above Points

  • Apply image processing to increase contrast and brightness
  • Capture a higher resolution input
  • Do matching process in Big Data tool
  • Detect and recognize face in multi orientation

YAVA for Face Recognition Has Advantages :

  • Multi-parallel recognition processing
  • Easy to Build and maintain
  • Faster matching process
  • A large and fast-accessing database to save and access pattern of objects
  • Provides deep learning analysis from recognized face
  • Secure

Implementation for Face Recognition

  • Preventing and eliminating crime
  • Catching thieves identity
  • Greeting guests
  • Taking attendance
  • Identifying lost children
  • Door key replacements

Technology

  • OpenCV
  • Structural similarity
  • MapReduce

Weather Modelling

Climate change due to global warming becomes a significant effect to earth condition. There are many factors affect on climate change, natural and un-natural factor. One kind of natural factor is rainfall rate. Rainfall rate in Indonesia is high and need a specific method to use it in weather modelling. Humidity and temperature are also parameter must be included in a weather modelling. Beside natural factors, any industrial development is one of unnatural factor that contribute to climate change. These parameters to observe make weather modelling become harder task. The natural parameters are observed using Weather FMCW radar made by Labs247 collaborating with our partner.

Existing Challenges to Deal with :

  • Many parameters to observe
  • Indonesia has a different weather and need a specific approach
  • Minor change of weather data need an accurate measurement sensor
  • Heavy weather data process
  • Necessity to extend radar scope observation to get more data

Methods to Address the Above Points

  • Use Ensemble Kalman Filter(EnKF) to observe various weather data
  • Develop a suitable weather model that suit for Indonesia weather (numerical weather prediction)
  • Do process in Apache Spark
  • Use high resolution and sensitivity polarimetric-radar
  • Increase radar transmit power to extend its maximum range

Benefit

  • Give accurate weather prediction
  • Meets observation need
  • The use of EnKF increase modelling accuracy so provide more accurate prediction
  • Help society to build better plan for their vacation
  • Make society more aware of any climate change and reduce global warming effect

YAVA for Weather Modelling Has Advantages :

  • Apache Spark simplify and speed up weather data process
  • Provide a fast and large database to access
  • Easy to build using drag and drop interface
  • Deep learning analysis to give more accurate prediction

Weather Modelling Implementation

  • Weather modelling that suit for Indonesian climate
  • Weather prediction
  • First step of any task that need weather data
  • An early warning system of any weather change

Technology

  • Ensemble Kalman Filter (EnKF)
  • HDFS
  • YARN
  • Apache Spark

Multi Target Tracking in Marine Surveillance System

In an advanced radar system, object tracking and detection as a must to support radar main function. Ship movement is a object parameter to observe for marine and coastal radar. Ship movement has its own characteristics that differ from another object movement, such as car and aircraft. Ships sail in low speed and its track has many distortion caused by sea currents and waves. Therefore ship movement need a specific approach to model it. Usually, Kalman Filter methods used to track and detect ship movement in radar display. Kalman Filter do those task by using some previous states.

Existing Challenges to Deal with :

  • Multiple target tracking (MTT) is a complex process, the complexity increased as the number of target (ship) increased
  • Data association process between prediction and real outcome movement spent much time
  • Separate two or more close targets
  • Sea current and waves disturb ship track

Methods to Address the Above Points

  • Use Ensemble Kalman Filter to get better next move ship prediction
  • Run data association process in Big Data tools
  • Apply image processing to image display radar
  • Use high resolution radar
  • Use a suitable prediction modelling of ship movement

Benefit

  • Provide accurate prediction of ship motion so can increase sailing safety
  • Give a warning whenever two or more ships will collide with each other
  • Pretend illegal fishing by detecting where the ship come from and analyze its movement
  • Reduce any crime in shipping transportation
  • Find lost ship

YAVA for Multi Target Tracking in Marine Surveillance System Has Advantages :

  • Support Multi-parallel process
  • Calculation finished faster, important for a large number of target tracking and detection process
  • Analyze ship movement using deep learning to provide more accurate prediction
  • Easy to build using drag and drop interface

Weather Modelling Implementation

  • Real-time multi target detection and tracking
  • As an advanced sea surveillance system
  • Provide estimation track of ships
  • Prevent collision between ships
  • Risk mitigation

Technology

  • Ensemble Kalman Filter
  • Apache Kafka
  • Apache Spark
  • Tensorflow
  • Druid
  • HGrid247

Sensor Fusion

In an integrated ship monitoring system, the use of multiple sensor is a must. Involvement of multiple sensors are very important for detection, identification, and categorization of moving objects. A complete ship monitoring system, at least must have four different sensors with its own functions, those are Radar, AIS Receiver, Long Range Camera, and Direction Finder. Each sensor provides different format and type of data.

Existing Challenges to Deal with :

  • Integrate all sensor
  • A large database to save data collected from sensors
  • A processing computing device to handle all incoming data from various sensor
  • Comparing and choosing data between same kind sensor
  • The need of neural network topology to model, predict, and categorize various incoming data

Methods to Address the Above Points

  • Run multi-parallel processing to handle incoming data from many sensors
  • Do process in Hadoop
  • Apply Kalman filter to the same kind data and choose which data to use
  • Use deep learning approach to deal with all sensors data and integrate them

Benefit

  • Give complete information of sea traffic
  • Provide a more accurate and reliable prediction by integrating all data sensing
  • Help government to monitor the transportation sea and protect our sea resource
  • Reduce ship collision accident caused by inaccuracies of sensing device

YAVA for Sensor Fusion Has Advantages :

  • Support Multi-parallel process
  • Can handle all incoming sensor data easily
  • Integrate and analyze all sensing datas using deep learning to provide more accurate prediction
  • Easy to build using drag and drop interface

Weather Modelling Implementation

  • Provide many data from various sensor
  • Input of monitoring ship system
  • Reduce uncertainty of sensor measurement
  • Risk mitigation

Technology

  • Ensemble Kalman Filter
  • Apache Kafka
  • Apache Spark
  • Tensorflow
  • Druid
  • HGrid247
YAVA - BIG DATA SOLUTION WITHIN YOUR REACH

© 2017 Labs247. All rights reserved.
YAVA logo and HGrid247 logo are registered trademarks or trademarks of the Labs247 Company.
HADOOP, the Hadoop Elephant Logo, Apache, Flume, Ambari, Yarn, Bigtop, Phoenix, Hive, Tez, Oozie, HBase, Mahout, Pig, Solr, Storm, Spark, Sqoop, Impala, and ZooKeeper are registered trademarks or trademarks of the Apache Software Foundation.