Kevin Mellott is a Team Lead and Spark Data Developer at FIS, working to improve the data processing pipeline of one of the world’s largest FinTech companies. Although he began as a traditional Software Engineer, his career ventured into the world of data science several years ago. Recent projects have included the use of Spark’s machine learning pipelines as well as the Python Natural Language Toolkit. When he isn’t wrestling with Big Data, Kevin can be found at the local movie theater or ice hockey rink.
April 24, 2019 05:00 PM PT
Life occurs in real-time, and not surprisingly, more solutions are being built using streaming technologies. Event-based architectures are becoming the norm, and customers are expecting immediate access to their data. This new world offers many exciting opportunities, but also some new challenges. What do you do when your streaming data is not complete? What if it relies on another data source? Does the dependent data exist yet, and does it come from a 3rd party? How do we merge a complete picture of data when data is sourcing from multiple places at the same time? A new norm in the world of distributed services.
Join us as we dive deep into the technical details around these scenarios and more. Expect to learn about stream-stream joins, enriching stream data using local or remote data, and ways to anticipate and correct errors within the stream. Leave with a better understanding of managing data dependencies within a Spark Structured Streaming application.
June 5, 2018 05:00 PM PT
A mobile application is only as good as our design and how customers use it. But how do they use it? We've got over 35 million devices running our mobile banking platform, and we need to understand each and every one of them. Is the customer enjoying their experience, are they lost, or are they a fraudulent hacker 3000 miles away?
We developed an algorithm to examine the user's workflow so we can perform near real-time analysis of their online activities. We leverage Spark's Structured Streaming, ML Pipelines & GraphFrames, and good old fashioned grit to gain insights that allow us to improve our mobile app. For good measure, we added fraud detection to the mix so we can use artificial intelligence to detect any strange or alarming patterns.
Session hashtag: #DevSAIS18