CARIAD, an automotive software and technology company, unites more than 6,000 global experts and aligns major brands in the Volkswagen Group under one software strategy. Founded in 2020, CARIAD provides solutions to securely and compliantly update the fleet from mere transport to fully integrated digital experiences. CARIAD’s use of Docker provides a framework for embedding advanced software into existing systems.
As a subsidiary of Volkswagen Group, CARIAD has expertise in complex identity access requirements, including integrating Docker with multiple Active Directory instances. Security and compliance requirements are critical, with added layers of complexity due to environment requirements introduced when developing embedded systems.
Docker Business is a specialized containerization platform for large enterprises, providing features that enhance security, compliance, and scalability. CARIAD leverages Docker Business to integrate Single Sign-On (SSO) and Image Access Management (IAM), which are crucial for meeting their stringent security requirements. These features allow CARIAD to control access to Docker resources effectively, supporting their security and compliance requirements.
Integration with WSL 2
Docker Desktop makes it simple for CARIAD developers to run Linux containers natively on their Windows machines without the need for a dual-boot setup or a dedicated Linux machine.
Windows Subsystem for Linux 2 (WSL 2) provides a hybrid development environment, with a Linux kernel running in a lightweight virtual machine, fully managed by Windows, yet offering near-native performance.
Before WSL 2, the original WSL used a translation layer between Windows and the Linux file system, which introduced potential performance bottlenecks, especially for running build scripts or version control operations. WSL 2 introduces a full Linux kernel with a real Linux file system, stored in a virtual disk image. This greatly improves file IO performance and supports a broader range of tools and applications with better Linux system call support.
WSL 2 also improves resource management by dynamically managing memory and CPU resources allocated to the Linux subsystem. This functionality is crucial for CARIAD because it allows efficient scaling of resources based on workload demands, which is particularly important when developing and testing resource-intensive applications.
Docker Desktop integrates well with WSL 2 and provides the capability to execute Docker commands with any Linux distribution installed within WSL 2. This approach enables CARIAD to execute Docker commands within a custom WSL distribution that adheres to their organizational policy requirements.
Single Sign-On and User Access Management
CARIAD integrates Docker SSO, available in Docker Business, with its existing Azure Active Directory instances to ensure that only authenticated and authorized users access Docker resources, aligning with required policies. Enhancing the benefits of Enterprise SSO, this feature is crucial for proper configuration and enforcement of other security measures, like Image Access Management (IAM).
Image Access Management
CARIAD ensures it uses only authorized images from Docker Hub, enforced through tailored administrative configurations with IAM. This approach manages access levels by group and is a key component in enforcing security protocols, particularly in safeguarding container environments. Properly configured and enforced IAM, which is automatically enabled by enforcing sign-in, reduces the risk associated with unauthorized or unsecured images.
This process involves activating IAM, setting permissions that align with user roles and project requirements, and testing to ensure the permissions are working as intended (Figure 1).
The CARIAD team explains the importance of RAM and IAM when using WSL 2 this way: “While WSL 2 seamlessly grants elevated root capabilities within its environment, it is fortunate that these permissions do not extend to SYSTEM rights on the Windows host. However, if both registry and image access management are absent by the Docker Desktop setup, the lack of firewall and anti-malware protection could introduce a potential malicious container attack and a local privilege escalation.”
Figure 1: Potential introduction of a malicious container.
Conclusion
CARIAD’s strategies for deploying Docker Business into a secure enterprise environment represent strong choices for any organization managing similar security, compliance, or identity access management requirements. For organizations looking to enhance their development operations, CARIAD’s model offers a blueprint for deploying Docker Desktop to large enterprises.
Using Docker Business features and WSL 2, CARIAD ensures compliance and supports a developer-friendly workflow. Within the stringent requirements necessary for automotive systems, developers at Volkswagen Group work with best-in-class tools and processes to build securely and quickly. CARIAD’s approach provides valuable lessons for enterprises looking to improve their development operations with Docker.
Videos are full of valuable information, but tools are often needed to help find it. From educational institutions seeking to analyze lectures and tutorials to businesses aiming to understand customer sentiment in video reviews, transcribing and understanding video content is crucial for informed decision-making and innovation. Recently, advancements in AI/ML technologies have made this task more accessible than ever.
Developing GenAI technologies with Docker opens up endless possibilities for unlocking insights from video content. By leveraging transcription, embeddings, and large language models (LLMs), organizations can gain deeper understanding and make informed decisions using diverse and raw data such as videos.
In this article, we’ll dive into a video transcription and chat project that leverages the GenAI Stack, along with seamless integration provided by Docker, to streamline video content processing and understanding.
High-level architecture
The application’s architecture is designed to facilitate efficient processing and analysis of video content, leveraging cutting-edge AI technologies and containerization for scalability and flexibility. Figure 1 shows an overview of the architecture, which uses Pinecone to store and retrieve the embeddings of video transcriptions.
Figure 1: Schematic diagram outlining a two-component system for processing and interacting with video data.
The application’s high-level service architecture includes the following:
yt-whisper: A local service, run by Docker Compose, that interacts with the remote OpenAI and Pinecone services. Whisper is an automatic speech recognition (ASR) system developed by OpenAI, representing a significant milestone in AI-driven speech processing. Trained on an extensive dataset of 680,000 hours of multilingual and multitask supervised data sourced from the web, Whisper demonstrates remarkable robustness and accuracy in English speech recognition.
Dockerbot: A local service, run by Docker Compose, that interacts with the remote OpenAI and Pinecone services. The service takes the question of a user, computes a corresponding embedding, and then finds the most relevant transcriptions in the video knowledge database. The transcriptions are then presented to an LLM, which takes the transcriptions and the question and tries to provide an answer based on this information.
OpenAI: The OpenAI API provides an LLM service, which is known for its cutting-edge AI and machine learning technologies. In this application, OpenAI’s technology is used to generate transcriptions from audio (using the Whisper model) and to create embeddings for text data, as well as to generate responses to user queries (using GPT and chat completions).
Pinecone: A vector database service optimized for similarity search, used for building and deploying large-scale vector search applications. In this application, Pinecone is employed to store and retrieve the embeddings of video transcriptions, enabling efficient and relevant search functionality within the application based on user queries.
The application is a chatbot that can answer questions from a video. Additionally, it provides timestamps from the video that can help you find the sources used to answer your question.
In the /docker-genai directory, create a text file called .env, and specify your API keys inside. The following snippet shows the contents of the .env.example file that you can refer to as an example.
#-------------------------------------------------------------
# OpenAI
#-------------------------------------------------------------
OPENAI_TOKEN=your-api-key # Replace your-api-key with your personal API key
#-------------------------------------------------------------
# Pinecone
#--------------------------------------------------------------
PINECONE_TOKEN=your-api-key # Replace your-api-key with your personal API key
Build and run the application
In a terminal, change directory to your docker-genai directory and run the following command:
docker compose up --build
Next, Docker Compose builds and runs the application based on the services defined in the docker-compose.yaml file. When the application is running, you’ll see the logs of two services in the terminal.
In the logs, you’ll see the services are exposed on ports 8503 and 8504. The two services are complementary to each other.
The yt-whisper service is running on port 8503. This service feeds the Pinecone database with videos that you want to archive in your knowledge database. The next section explores the yt-whisper service.
Using yt-whisper
The yt-whisper service is a YouTube video processing service that uses the OpenAI Whisper model to generate transcriptions of videos and stores them in a Pinecone database. The following steps outline how to use the service.
Open a browser and access the yt-whisper service at http://localhost:8503. Once the application appears, specify a YouTube video URL in the URL field and select Submit. The example shown in Figure 2 uses a video from David Cardozo.
Figure 2: A web interface showcasing processed video content with a feature to download transcriptions.
Submitting a video
The yt-whisper service downloads the audio of the video, then uses Whisper to transcribe it into a WebVTT (*.vtt) format (which you can download). Next, it uses the “text-embedding-3-small” model to create embeddings and finally uploads those embeddings into the Pinecone database.
After the video is processed, a video list appears in the web app that informs you which videos have been indexed in Pinecone. It also provides a button to download the transcript.
Accessing Dockerbot chat service
You can now access the Dockerbot chat service on port 8504 and ask questions about the videos as shown in Figure 3.
Figure 3: Example of a user asking Dockerbot about NVIDIA containers and the application giving a response with links to specific timestamps in the video.
Conclusion
In this article, we explored the exciting potential of GenAI technologies combined with Docker for unlocking valuable insights from video content. It shows how the integration of cutting-edge AI models like Whisper, coupled with efficient database solutions like Pinecone, empowers organizations to transform raw video data into actionable knowledge.
Whether you’re an experienced developer or just starting to explore the world of AI, the provided resources and code make it simple to embark on your own video-understanding projects.
Learning about and sharing ways in which those in the Docker community leverage Docker to drive innovation is always exciting. We are learning about many interesting AI/ML solutions that use Docker to accelerate development and simplify deployment.
In this blog post, Saul Martin shares how Prometeo.ai leverages Docker to deploy and manage its machine learning models allowing its developers to focus on innovation, not infrastructure/deployment management, for their sentiment analysis of a range of cryptocurrencies to provide insights for traders.
The digital assets market, which is famously volatile and swift, necessitates tools that can keep up with its speed and provide real-time insights. At the forefront of these tools is Prometeo.ai, which has harnessed the power of Docker to build a sophisticated, high-frequency sentiment analysis platform. This tool sifts through the torrent of emotions that drive the cryptocurrency market, providing real-time sentiments of the top 100 assets, which is a valuable resource for hedge funds and financial institutions.
Prometeo.ai’s leveraging of Docker’s containerization capabilities allows it to deploy and manage complex machine learning models with ease, making it an example of modern, robust, scalable architecture.
In this blog post, we will delve into how Prometeo.ai is utilizing Docker for its sentiment analysis tool, highlighting the critical aspects of its data collection, machine learning model implementations, storage, and deployment processes. This exploration will give you a clear understanding of how Docker can transform machine learning application deployment, presenting a case study in the form of Prometeo.ai.
Data collection and processing: High-frequency sentiment analysis with Docker
Prometeo.ai’s comprehensive sentiment analysis capability hinges on an agile, near real-time data collection and processing infrastructure. This framework captures, enriches, and publishes an extensive range of sentiment data from diverse platforms:
Stream/data access: Platform-specific data pipelines tasked with real-time harvesting of cryptocurrency-related discussions hosted on siloed Docker containers.
Tokenization and sentiment analysis: The harvested data undergoes tokenization, transforming each content piece into a format suitable for analysis. An internal Sentiment Analysis API further enriches this tokenized data, inferring sentiment attributes from the raw information.
Publishing: Enriched sentiment data is published within one minute of collection, facilitating near real-time insights for users. During periods of content unavailability from a data source, the system generates and publishes an empty dictionary.
All these operations transpire within Docker containers, guaranteeing the necessary scalability, isolation, and resource efficiency to manage high-frequency data operations.
For efficient data storage, Prometeo.ai relies on:
NoSQL database: DynamoDB is used for storing minute-level sentiment aggregations. The primary key is defined such that it allows quick access to data based on time-range queries. These aggregations are critical for providing real-time insights to users and for hourly and daily aggregation calculations.
Object storage: For model retraining and data backup purposes, the raw data, including raw content, is exported in batches and stored in Amazon S3 buckets. This robust storage mechanism ensures data durability and aids in maintaining data integrity.
Relational database: Metadata related to different assets, including links, tickers, IDs, descriptions, and others, are hosted in PostgreSQL. This provides a relational structure for asset metadata and promotes accessible, structured access when required.
NLP models
Prometeo.ai makes use of two Bidirectional Encoder Representations from Transformers (BERT) models, both of which operate within a Docker environment for natural language processing (NLP). The following models run multi-label classification pipelines that have been fine-tuned on an in-house dataset of 50k manually labeled tweets.
proSENT model: This model specializes in identifying 28 unique emotional sentiments. It owes its comprehensive language understanding to training on a corpus of more than 1.5 million unique cryptocurrency-related social media posts.
proCRYPT model: This model is fine-tuned for crypto sentiment analysis, classifying sentiments as bullish, bearish, or neutral. The deployment architecture encapsulates both these models within a Docker container alongside a FastAPI server. This internal API acts as the conduit for running inferences.
To ensure a seamless and efficient build process, Hugging Face’s model hub is used to store the models. The models and their binary files are retrieved directly from Hugging Face during the Docker container’s build phase. This approach keeps the Docker container lean by downloading the models at runtime, thereby optimizing the build process and contributing to the overall operational efficiency of the application.
Deployment
Prometeo.ai’s deployment pipeline is composed of GitHub Actions, AWS CodeDeploy, and accelerated computing instances. This pipeline forms a coherent system for efficiently handling application updates and resource allocation:
GitHub Actions: The onset of the pipeline employs GitHub Actions, which are programmed to autonomously instigate a fresh deployment upon the push of any modifications to the production branch. This procedural design ensures the application continually operates on the most recent, vetted code version.
AWS CodeDeploy: The subsequent phase involves AWS CodeDeploy, which is triggered once GitHub Actions have successfully compiled and transferred the Docker image to the Elastic Container Registry (ECR). CodeDeploy is tasked with the automatic deployment of this updated Docker image to the GPU-optimized instances. This robust orchestration ensures smooth rollouts and establishes a reliable rollback plan if necessary.
Accelerated computing: Prometeo leverages NVIDIA Tesla GPUs for the computational prowess needed for executing complex BERT models. These GPU-optimized instances are tailored for NVIDIA-CUDA Docker image compatibility, thereby facilitating GPU acceleration, which significantly expedites the processing and analysis stages.
Below is a snippet demonstrating the configuration to exploit the GPU capabilities of the instances:
Please note that the image must be the same version as your CUDA version after running nvidia-smi:
FROM nvidia/cuda:12.1.0-base-ubuntu20.04
To maintain optimal performance under fluctuating load conditions, an autoscaling mechanism is incorporated. This solution perpetually monitors CPU utilization, dynamically scaling the number of instances up or down as dictated by the load. This ensures that the application always has access to the appropriate resources for efficient execution.
Conclusion
By harnessing Docker’s containerization capabilities and compatibility with NVIDIA-CUDA images, Prometeo.ai successfully manages intensive, real-time emotion analysis in the digital assets domain. Docker’s role in this strategy is pivotal, providing an environment that enables resource optimization and seamless integration with other services.
Prometeo.ai’s implementation demonstrates Docker’s potential to handle sophisticated computational tasks. The orchestration of Docker with GPU-optimized instances and cloud-based services exhibits a scalable and efficient infrastructure for high-frequency, near-real-time data analysis.
Do you have an interesting use case or story about Docker in your AI/ML workflow? We would love to hear from you and maybe even share your story.
JupyterLab is an open source application built around the concept of a computational notebook document. It enables sharing and executing code, data processing, visualization, and offers a range of interactive features for creating graphs.
The latest version, JupyterLab 4.0, was released in early June. Compared to its predecessors, this version features a faster Web UI, improved editor performance, a new Extension Manager, and real-time collaboration.
If you have already installed the standalone 3.x version, evaluating the new features will require rewriting your current environment, which can be labor-intensive and risky. However, in environments where Docker operates, such as Docker Desktop, you can start an isolated JupyterLab 4.0 in a container without affecting your installed JupyterLab environment. Of course, you can run these without impacting the existing environment and access them on a different port.
In this article, we show how to quickly evaluate the new features of JupyterLab 4.0 using Jupyter Docker Stacks on Docker Desktop, without affecting the host PC side.
Why containerize JupyterLab?
Users have downloaded the base image of JupyterLab Notebook stack Docker Official Image more than 10 million times from Docker Hub. What’s driving this significant download rate? There’s an ever-increasing demand for Docker containers to streamline development workflows, while allowing JupyterLab developers to innovate with their choice of project-tailored tools, application stacks, and deployment environments. Our JupyterLab notebook stack official image also supports both AMD64 and Arm64/v8 platforms.
Containerizing the JupyterLab environment offers numerous benefits, including the following:
Containerization ensures that your JupyterLab environment remains consistent across different deployments. Whether you’re running JupyterLab on your local machine, in a development environment, or in a production cluster, using the same container image guarantees a consistent setup. This approach helps eliminate compatibility issues and ensures that your notebooks behave the same way across different environments.
Packaging JupyterLab in a container allows you to easily share your notebook environment with others, regardless of their operating system or setup. This eliminates the need for manually installing dependencies and configuring the environment, making it easier to collaborate and share reproducible research or workflows. And this is particularly helpful in AI/ML projects, where reproducibility is crucial.
Containers enable scalability, allowing you to scale your JupyterLab environment based on the workload requirements. You can easily spin up multiple containers running JupyterLab instances, distribute the workload, and take advantage of container orchestration platforms like Kubernetes for efficient resource management. This becomes increasingly important in AI/ML development, where resource-intensive tasks are common.
Getting started
To use JupyterLab on your computer, one option is to use the JupyterLab Desktop application. It’s based on Electron, so it operates with a GUI on Windows, macOS, and Linux. Indeed, using JupyterLab Desktop makes the installation process fairly simple. In a Windows environment, however, you’ll also need to set up the Python language separately, and, to extend the capabilities, you’ll need to use pip to set up packages.
Although such a desktop solution may be simpler than building from scratch, we think the combination of Docker Desktop and Docker Stacks is still the more straightforward option. With JupyterLab Desktop, you cannot mix multiple versions or easily delete them after evaluation. Above all, it does not provide a consistent user experience across Windows, macOS, and Linux.
On a Windows command prompt, execute the following command to launch a basic notebook:
docker container run -it --rm -p 10000:8888 jupyter/base-notebook
This command utilizes the jupyter/base-notebook Docker image, maps the host’s port 10000 to the container’s port 8888, and enables command input and a pseudo-terminal. Additionally, an option is added to delete the container once the process is completed.
After waiting for the Docker image to download, access and token information will be displayed on the command prompt as follows. Here, rewrite the URL http://127.0.0.1:8888 to http://127.0.0.1:10000 and then append the token to the end of this URL. In this example, the output will look like this:
Note that this token is specific to my environment, so copying it will not work for you. You should replace it with the one actually displayed on your command prompt.
Then, after waiting for a short while, JupyterLab will launch (Figure 1). From here, you can start a Notebook, access Python’s console environment, or utilize other work environments.
Figure 1. The page after entering the JupyterLab token. The left side is a file list, and the right side allows you to open Notebook creation, Python console, etc.
The port 10000 on the host side is mapped to port 8888 inside the container, as shown in Figure 2.
Figure 2. The host port 10000 is mapped to port 8888 inside the container.
In the Password or token input form on the screen, enter the token displayed in the command line or in the container logs (the string following token=), and select Log in, as shown in Figure 3.
Figure 3. Enter the token that appears in the container logs.
By the way, in this environment, the data will be erased when the container is stopped. If you want to reuse your data even after stopping the container, create a volume by adding the -v option when launching the Docker container.
To stop this container environment, click CTRL-C on the command prompt, then respond to the Jupyter server’s prompt Shutdown this Jupyter server (y/[n])? with y and press enter. If you are using Docker Desktop, stop the target container from the Containers.
Shutdown this Jupyter server (y/[n])? y
[C 2023-06-26 01:39:52.997 ServerApp] Shutdown confirmed
[I 2023-06-26 01:39:52.998 ServerApp] Shutting down 5 extensions
[I 2023-06-26 01:39:52.998 ServerApp] Shutting down 1 kernel
[I 2023-06-26 01:39:52.998 ServerApp] Kernel shutdown: 653f7c27-03ff-4604-a06c-2cb4630c098d
Once the display changes as follows, the container is terminated and the data is deleted.
When the container is running, data is saved in the /home/jovyan/work/ directory inside the container. You can either bind mount this as a volume or allocate it as a volume when starting the container. By doing so, even if you stop the container, you can use the same data again when you restart the container:
Note: The \ symbol signifies that the command line continues on the command prompt. You may also write the command in a single line without using the \ symbol. However, in the case of Windows command prompt, you need to use the ^ symbol instead.
With this setup, when launched, the JupyterLab container mounts the /work/ directory to the folder where the docker container run command was executed. Because the data persists even when the container is stopped, you can continue using your Notebook data as it is when you start the container again.
Plotting using the famous Iris flower dataset
In the following example, we’ll use the Iris flower dataset, which consists of 150 records in total, with 50 samples from each of three types of Iris flowers (Iris setosa, Iris virginica, Iris versicolor). Each record consists of four numerical attributes (sepal length, sepal width, petal length, petal width) and one categorical attribute (type of iris). This data is included in the Python library scikit-learn, and we will use matplotlib to plot this data.
When trying to input the sample code from the scikit-learn page (the code is at the bottom of the page, and you can copy and paste it) into iPython, the following error occurs (Figure 4).
Figure 4. Error message occurred due to missing “matplotlib” module.
This is an error message on iPython stating that the “matplotlib” module does not exist. Additionally, the “scikit-learn” module is needed.
To avoid these errors and enable plotting, run the following command. Here, !pip signifies running the pip command within the iPython environment:
!pip install matplotlib scikit-learn
By pasting and executing the earlier sample code in the next cell on iPython, you can plot and display the Iris dataset as shown in Figure 5.
Figure 5. When the sample code runs successfully, two images will be output.
Note that it can be cumbersome to use the !pip command to add modules every time. Fortunately, you can add also add modules in the following ways:
By creating a dedicated Dockerfile
By using an existing group of images called Jupyter Docker Stacks
Building a Docker image
If you’re familiar with Dockerfile and building images, this five-step method is easy. Also, this approach can help keep the Docker image size in check.
Step 1. Creating a directory
To build a Docker image, the first step is to create and navigate to the directory where you’ll place your Dockerfile and context:
mkdir myjupyter && cd myjupyter
Step 2. Creating a requirements.txt file
Create a requirements.txt file and list the Python modules you want to add with the pip command:
matplotlib
scikit-learn
Step 3. Writing a Dockerfile
FROM jupyter/base-notebook
COPY ./requirements.txt /home/jovyan/work
RUN python -m pip install --no-cache -r requirements.txt
This Dockerfile specifies a base image jupyter/base-notebook, copies the requirements.txt file from the local directory to the /home/jovyan/work directory inside the container, and then runs a pip install command to install the Python packages listed in the requirements.txt file.
The docker run command instructs Docker to run a container.
The -it option attaches an interactive terminal to the container.
The -p 10000:8888 maps port 10000 on the host machine to port 8888 inside the container. This allows you to access Jupyter Notebook running in the container via http://localhost:10000 in your web browser.
The -v "%cd%":/home/jovyan/work mounts the current directory (%cd%) on the host machine to the /home/jovyan/work directory inside the container. This enables sharing files between the host and the Jupyter Notebook.
In this example, myjupyter is the name of the Docker image you want to run. Make sure you have the appropriate image available on your system. The operation after startup is the same as before. You don’t need to add libraries with the !pip command because the necessary libraries are included from the start.
How to use Jupyter Docker Stacks’ images
To execute the JupyterLab environment, we will utilize a Docker image called jupyter/scipy-notebook from the Jupyter Docker Stacks. Please note that the running Notebook will be terminated. After entering Ctrl-C on the command prompt, enter y and specify the running container.
This command will run a container using the jupyter/scipy-notebook image, which provides a Jupyter Notebook environment with additional scientific libraries.
Here’s a breakdown of the command:
The docker run command starts a new container.
The -it option attaches an interactive terminal to the container.
The -p 10000:8888 maps port 10000 on the host machine to port 8888 inside the container, allowing access to Jupyter Notebook at http://localhost:10000.
The -v "$(pwd)":/home/jovyan/work mounts the current directory ($(pwd)) on the host machine to the /home/jovyan/work directory inside the container. This enables sharing files between the host and the Jupyter Notebook.
The jupyter/scipy-notebook is the name of the Docker image used for the container. Make sure you have this image available on your system.
The previous JupyterLab image was a minimal Notebook environment. The image we are using this time includes many packages used in the scientific field, such as numpy and pandas, so it may take some time to download the Docker image. This one is close to 4GB in image size.
Once the container is running, you should be able to run the Iris dataset sample immediately without having to execute pip like before. Give it a try.
Some images include TensorFlow’s deep learning library, ones for the R language, Julia programming language, and Apache Spark. See the image list page for details.
In a Windows environment, you can easily run and evaluate the new version of JupyterLab 4.0 using Docker Desktop. Doing so will not affect or conflict with the existing Python language environment. Furthermore, this setup provides a consistent user experience across other platforms, such as macOS and Linux, making it the ideal solution for those who want to try it.
Conclusion
By containerizing JupyterLab with Docker, AI/ML developers gain numerous advantages, including consistency, easy sharing and collaboration, and scalability. It enables efficient management of AI/ML development workflows, making it easier to experiment, collaborate, and reproduce results across different environments. With JupyterLab 4.0 and Docker, the possibilities for supercharging your AI/ML development are limitless. So why wait? Embrace containerization and experience the true power of JupyterLab in your AI/ML projects.
Guest author Amin Choroomi is an experienced software developer at Kinsta. Passionate about Docker and Kubernetes, he specializes in application development and DevOps practices. His expertise lies in leveraging these transformative technologies to streamline deployment processes and enhance software scalability.
One of the biggest challenges of developing and maintaining cloud-native applications at the enterprise level is having a consistent experience through the entire development lifecycle. This process is even harder for remote companies with distributed teams working on different platforms, with different setups, and asynchronous communication.
At Kinsta, we have projects of all sizes for application hosting, database hosting, and managed WordPress hosting. We need to provide a consistent, reliable, and scalable solution that allows:
Developers and quality assurance teams, regardless of their operating systems, to create a straightforward and minimal setup for developing and testing features.
DevOps, SysOps, and Infrastructure teams to configure and maintain staging and production environments.
Overcoming the challenge of developing cloud-native applications on a distributed team
At Kinsta, we rely heavily on Docker for this consistent experience at every step, from development to production. In this article, we’ll walk you through:
How to leverage Docker Desktop to increase developers’ productivity.
How we build Docker images and push them to Google Container Registry via CI pipelines with CircleCI and GitHub Actions.
How we use CD pipelines to promote incremental changes to production using Docker images, Google Kubernetes Engine, and Cloud Deploy.
How the QA team seamlessly uses prebuilt Docker images in different environments.
Using Docker Desktop to improve the developer experience
Running an application locally requires developers to meticulously prepare the environment, install all the dependencies, set up servers and services, and make sure they are properly configured. When you run multiple applications, this approach can be cumbersome, especially when it comes to complex projects with multiple dependencies. And, when you introduce multiple contributors with multiple operating systems, chaos is installed. To prevent this, we use Docker.
With Docker, you can declare the environment configurations, install the dependencies, and build images with everything where it should be. Anyone, anywhere, with any OS can use the same images and have exactly the same experience as anyone else.
Declare your configuration with Docker Compose
To get started, you need to create a Docker Compose file, docker-compose.yml. This is a declarative configuration file written in YAML format that tells Docker your application’s desired state. Docker uses this information to set up the environment for your application.
Docker Compose files come in handy when you have more than one container running and there are dependencies between containers.
To create your docker-compose.yml file:
Start by choosing an image as the base for our application. Search on Docker Hub to find a Docker image that already contains your app’s dependencies. Make sure to use a specific image tag to avoid errors. Using the latest tag can cause unforeseen errors in your application. You can use multiple base images for multiple dependencies — for example, one for PostgreSQL and one for Redis.
Use volumes to persist data on your host if you need to. Persisting data on the host machine helps you avoid losing data if Docker containers are deleted or if you have to recreate them.
Use networks to isolate your setup to avoid network conflicts with the host and other containers. It also helps your containers to find and communicate with each other easily.
Bringing it all together, we have a docker-compose.yml that looks like this:
To begin, we need to build a Docker image using a Dockerfile, and then call that from docker-compose.yml.
Follow these five steps to create your Dockerfile file:
1. Start by choosing an image as a base. Use the smallest base image that works for the app. Usually, alpine images are minimal with nearly zero extra packages installed. You can start with an alpine image and build on top of that:
docker
FROM node:18.15.0-alpine3.17
2. Sometimes you need to use a specific CPU architecture to avoid conflicts. For example, suppose that you use an arm64-based processor but you need to build an amd64 image. You can do that by specifying the -- platform in Dockerfile:
docker
FROM --platform=amd64 node:18.15.0-alpine3.17
3. Define the application directory and install the dependencies and copy the output to your root directory:
5. Implement auto-reload so that when you change something in the source code, you can preview your changes immediately without having to rebuild the application manually. To do that, build the image first, then run it in a separate service:
Pro tip: Note that node_modules is also mounted explicitly to avoid platform-specific issues with packages. This means that, instead of using the node_modules on the host, the Docker container uses its own but maps it on the host in a separate volume.
Incrementally build the production images with continuous integration
The majority of our apps and services use CI/CD for deployment, and Docker plays an important role in the process. Every change in the main branch immediately triggers a build pipeline through either GitHub Actions or CircleCI. The general workflow is simple: It installs the dependencies, runs the tests, builds the Docker image, and pushes it to Google Container Registry (or Artifact Registry). In this article, we’ll describe the build step.
Building the Docker images
We use multi-stage builds for security and performance reasons.
Stage 1: Builder
In this stage, we copy the entire code base with all source and configuration, install all dependencies, including dev dependencies, and build the app. It creates a dist/ folder and copies the built version of the code there. This image is way too large, however, with a huge set of footprints to be used for production. Also, as we use private NPM registries, we use our private NPM_TOKEN in this stage as well. So, we definitely don’t want this stage to be exposed to the outside world. The only thing we need from this stage is the dist/ folder.
Stage 2: Production
Most people use this stage for runtime because it is close to what we need to run the app. However, we still need to install production dependencies, and that means we leave footprints and need the NPM_TOKEN. So, this stage is still not ready to be exposed. Here, you should also note the yarn cache clean on line 19. That tiny command cuts our image size by up to 60 percent.
Stage 3: Runtime
The last stage needs to be as slim as possible with minimal footprints. So, we just copy the fully baked app from production and move on. We put all those yarn and NPM_TOKEN stuff behind and only run the app.
This is the final Dockerfile.production:
docker
# Stage 1: build the source code
FROM node:18.15.0-alpine3.17 as builder
WORKDIR /opt/app
COPY package.json yarn.lock ./
RUN yarn install
COPY . .
RUN yarn build
# Stage 2: copy the built version and build the production dependencies FROM node:18.15.0-alpine3.17 as production
WORKDIR /opt/app
COPY package.json yarn.lock ./
RUN yarn install --production && yarn cache clean
COPY --from=builder /opt/app/dist/ ./dist/
# Stage 3: copy the production ready app to runtime
FROM node:18.15.0-alpine3.17 as runtime
WORKDIR /opt/app
COPY --from=production /opt/app/ .
CMD ["yarn", "start"]
Note that, for all the stages, we start copying package.json and yarn.lock files first, installing the dependencies, and then copying the rest of the code base. The reason for this is that Docker builds each command as a layer on top of the previous one, and each build could use the previous layers if available and only build the new layers for performance purposes.
Let’s say you have changed something in src/services/service1.ts without touching the packages. That means the first four layers of the builder stage are untouched and could be reused. This approach makes the build process incredibly faster.
Pushing the app to Google Container Registry through CircleCI pipelines
There are several ways to build a Docker image in CircleCI pipelines. In our case, we chose to use circleci/gcp-gcr orbs:
Minimum configuration is needed to build and push our app, thanks to Docker.
With every small change pushed to the main branch, we build and push a new Docker image to the registry.
Deploying changes to Google Kubernetes Engine using Google Delivery Pipelines
Having ready-to-use Docker images for each and every change also makes it easier to deploy to production or roll back in case something goes wrong. We use Google Kubernetes Engine to manage and serve our apps, and we use Google Cloud Deploy and Delivery Pipelines for our continuous deployment process.
When the Docker image is built after each small change (with the CI pipeline shown previously), we take one step further and deploy the change to our dev cluster using gcloud. Let’s look at that step in CircleCI pipeline:
This step triggers a release process to roll out the changes in our dev Kubernetes cluster. After testing and getting the approvals, we promote the change to staging and then production. This action is all possible because we have a slim isolated Docker image for each change that has almost everything it needs. We only need to tell the deployment which tag to use.
How the Quality Assurance team benefits from this process
The QA team needs mostly a pre-production cloud version of the apps to be tested. However, sometimes they need to run a prebuilt app locally (with all the dependencies) to test a certain feature. In these cases, they don’t want or need to go through all the pain of cloning the entire project, installing npm packages, building the app, facing developer errors, and going over the entire development process to get the app up and running.
Now that everything is already available as a Docker image on Google Container Registry, all the QA team needs is a service in Docker compose file:
With this service, the team can spin up the application on their local machines using Docker containers by running:
docker compose up
This is a huge step toward simplifying testing processes. Even if QA decides to test a specific tag of the app, they can easily change the image tag on line 6 and re-run the Docker compose command. Even if they decide to compare different versions of the app simultaneously, they can easily achieve that with a few tweaks. The biggest benefit is to keep our QA team away from developer challenges.
Advantages of using Docker
Almost zero footprints for dependencies: If you ever decide to upgrade the version of Redis or PostgreSQL, you can just change one line and re-run the app. There’s no need to change anything on your system. Additionally, if you have two apps that both need Redis (maybe even with different versions) you can have both running in their own isolated environment, without any conflicts with each other.
Multiple instances of the app: There are many cases where we need to run the same app with a different command, such as initializing the DB, running tests, watching DB changes, or listening to messages. In each of these cases, because we already have the built image ready, we just add another service to the Docker compose file with a different command, and we’re done.
Easier testing environment: More often than not, you just need to run the app. You don’t need the code, the packages, or any local database connections. You only want to make sure the app works properly, or need a running instance as a backend service while you’re working on your own project. That could also be the case for QA, Pull Request reviewers, or even UX folks who want to make sure their design has been implemented properly. Our Docker setup makes it easy for all of them to take things going without having to deal with too many technical issues.
Connexion
