27-29 November, Vilnius
Conference about Big Data, High Load, Data Science, Machine Learning & AI
Conference is over. See you next year!
Lennard Cornelis is a Senior Big Data Engineer whohas a great passion for technology. He is really a hands-on person and love to solve difficult and challenging problems. Always in to learn new skills and loving the way how fast things are changing in this industry. Knowledge sharing is very important to him as he loves the role of mentoring colleagues. My latest challenge is to know everything about Docker.
Docker Data Science Pipeline
At ING, Cornelis and his team needed a way to implement Data science models from exploration into production. He will do this talk from his experience on the exploration and production Hadoop environment as a senior Ops engineer. For this they are using OpenShift to run Docker containers that connect to the big data Hadoop environment.
During this talk he will explain why they need this and how this is done at ING. Also how to set up a docker container running a data science model using Hive, Python, and Spark. He’ll also explain how to use Docker files to build Docker images, add all the needed components inside the Docker image, and how to run different versions of software in different containers.
In the end, he will also give a demo of how it runs and is automated using Git with webhook connecting to Jenkins and start the docker service that will connect to a big data Hadoop environment.
This is going to be a great technical talk for engineers and data scientist.