Containerized application development has revolutionized modern software delivery, but slow image builds in CI/CD pipelines can bring developer productivity to a halt. Even with AWS CodeBuild automating application testing and building, teams face challenges like resource constraints, inefficient caching, and complex multi-architecture builds that lead to delays, lower release frequency, and prolonged recovery times.
Enter Docker Build Cloud, a high-performance cloud service designed to streamline image builds, integrate seamlessly with AWS CodeBuild, and reduce build times dramatically. With Docker Build Cloud, you gain powerful cloud-based builders, shared caching, and native multi-architecture support — all while keeping your CI/CD pipelines efficient and your developers focused on delivering value faster.
In this post, we’ll explore how AWS CodeBuild combined with Docker Build Cloud tackles common bottlenecks, boosts build performance, and simplifies workflows, enabling teams to ship more quickly and reliably.
By using AWS CodeBuild, you can automate the build and testing of container applications, enabling the construction of efficient CI/CD workflows. AWS CodeBuild is also integrated with AWS Identity and Access Management (IAM), allowing detailed configuration of access permissions for build processes and control over AWS resources.
Container images built with AWS CodeBuild can be stored in Amazon Elastic Container Registry (Amazon ECR) and deployed to various AWS services, such as Amazon Elastic Container Service (Amazon ECS), Amazon Elastic Kubernetes Service (Amazon EKS), AWS Fargate, or AWS Lambda (Figure 1). Additionally, these services can leverage AWS Graviton, which adopts Arm-based architectures, to improve price performance for compute workloads.
Figure 1: CI/CD pipeline for AWS ECS using AWS CodeBuild (ECS Workshop).
Challenges of container image builds with AWS CodeBuild
Regardless of the tool used, building container images in a CI pipeline often takes a significant amount of time. This can lead to the following issues:
Reduced development productivity
Lower release frequency
Longer recovery time in case of failures
The main reasons why build times can be extended include:
1. Machines for building
Building container images requires substantial resources (CPU, RAM). If the machine specifications used in the CI pipeline are inadequate, build times can increase.
For simple container image builds, the impact may be minimal, but in cases of multi-stage builds or builds with many dependencies, the effect can be significant.
AWS CodeBuild allows changing instance types to improve these situations. However, such changes can apply to parts of the pipeline beyond container image builds, and they also increase costs.
Developers need to balance cost and build speed to optimize the pipeline.
2. Container image cache
In local development environments, Docker’s build cache can shorten rebuild times significantly by reusing previously built layers, avoiding redundant processing for unchanged parts of the Dockerfile. However, in cloud-based CI services, clean environments are used by default, so cache cannot be utilized, resulting in longer build times.
Although there are ways to use storage or container registries to leverage caching, these often are not employed because they introduce complexity in configuration and overhead from uploading and downloading cache data.
3. Multi-architecture builds (AMD64, Arm64)
To use Arm-based architectures like AWS Graviton in Amazon EKS or Amazon ECS, Arm64-compatible container image builds are required.
With changes in local environments, such as Apple Silicon, cases requiring multi-architecture support for AMD64 and Arm64 have increased. However, building images for different architectures (for example, building x86 on Arm, or vice versa) often requires emulation, which can further increase build times (Figure 2).
Although AWS CodeBuild provides both AMD64 and Arm64 instances, running them as separate pipelines is necessary, leading to more complex configurations and operations.
Figure 2: Creating multi-architecture Docker images using AWS CodeBuild.
Accelerating container image builds with Docker Build Cloud
The Docker Build Cloud service executes the Docker image build process in the cloud, significantly reducing build time and improving developer productivity (Figure 3).
Figure 3: How Docker Build Cloud works.
Particularly in CI pipelines, Docker Build Cloud enables faster container image builds without the need for significant changes or migrations to existing pipelines.
Docker Build Cloud includes the following features:
High-performance cloud builders: Cloud builders equipped with 16 vCPUs and 32GB RAM are available. This allows for faster builds compared to local environments or resource-constrained CI services.
Shared cache utilization: Cloud builders come with 200 GiB of shared cache, significantly reducing build times for subsequent builds. This cache is available without additional configuration, and Docker Build Cloud handles the cache maintenance for you.
Multi-architecture support (AMD64, Arm64): Docker Build Cloud supports native builds for multi-architecture with a single command. By specifying --platform linux/amd64,linux/arm64 in the docker buildx build command or using Bake, images for both Arm64 and AMD64 can be built simultaneously. This approach eliminates the need to split the pipeline for different architectures.
Architecture of AWS CodeBuild + Docker Build Cloud
Figure 4 shows an example of how to use Docker Build Cloud to accelerate container image builds in AWS CodeBuild:
Once the builder is successfully created, a guide is displayed for using it in local environments (Docker Desktop, CLI) or CI/CD environments (Figure 5).
Figure 5: Setup instructions of Docker Build Cloud.
Additionally, to use Docker Build Cloud from AWS CodeBuild, a Docker personal access token (PAT) is required. Store this token in AWS Secrets Manager for secure access.
Setting up the AWS CodeBuild pipeline
Next, set up the AWS CodeBuild pipeline. You should prepare an Amazon ECR repository to store the container images beforehand.
The following settings are used to create the AWS CodeBuild pipeline:
AMD64 instance with 3GB memory and 2 vCPUs.
Service role with permissions to push to Amazon ECR and access the Docker personal access token from AWS Secrets Manager.
In the install phase, Buildx, which is necessary for using Docker Build Cloud, is installed.
Although Buildx may already be installed in AWS CodeBuild, it might be an unsupported version for Docker Build Cloud. Therefore, it is recommended to install the latest version.
In the pre_build phase, the following steps are performed:
Log in to Amazon ECR.
Log in to Docker (Build Cloud).
Specify the cloud builder.
In the build phase, the image tag is specified, and the container image is built and pushed to Amazon ECR.
Instead of separating the build and push commands, using --push to directly push the image to Amazon ECR helps avoid unnecessary file transfers, contributing to faster builds.
Results comparison
To make a comparison, an AWS CodeBuild pipeline without Docker Build Cloud is created. The same instance type (AMD64, 3GB memory, 2vCPU) is used, and the build is limited to AMD64 container images.
Additionally, Docker login is used to avoid the pull rate limit imposed by Docker Hub.
Figure 6: The result of the execution without Docker Build Cloud.
Figure 7 shows the execution result of the AWS CodeBuild pipeline using Docker Build Cloud:
Figure 7: The result of the execution with Docker Build Cloud.
The results may vary depending on the container images being built and the state of the cache, but it was possible to build container images much faster and achieve multi-architecture builds (AMD64 and Arm64) within a single pipeline.
Conclusion
Integrating Docker Build Cloud into a CI/CD pipeline using AWS CodeBuild can dramatically reduce build times and improve release frequency. This allows developers to maximize productivity while delivering value to users more quickly.
As mentioned previously, the new Docker subscription already includes a free tier for Docker Build Cloud. Take advantage of this opportunity to test how much faster you can build container images for your current projects.
Effective infrastructure management is crucial for organizations using Docker Hub. Without a clear understanding of resource consumption, unexpected usage can emerge and skyrocket. This is particularly true if pulls and storage needs are not budgeted and forecasted correctly. By implementing proactive post controls and monitoring usage patterns, development teams can sustain their Docker Hub usage while keeping expenses under control.
To support these goals, we’ve introduced new Docker Hub Usage dashboards, offering organizations the ability to access and analyze their usage patterns for storage and pulls.
Docker Hub’s Usage dashboards put you in control, giving visibility into every pull and image your Docker systems request. Each pull and cache becomes a deliberate choice — not a random event — so you can make every byte count. With clear insights into what’s happening and why, you can design more efficient, optimized systems.
Reclaim control and manage technical resources by kicking bad habits
Figure 1: Docker Hub Usage dashboards.
The Docker Hub Usage dashboards (Figure 1) provide valuable insights, allowing teams to track peaks and valleys, detect high usage periods, and identify the images and repositories driving the most consumption. This visibility not only aids in managing usage but also strengthens continuous improvement efforts across your software supply chain, helping teams build applications more efficiently and sustainably.
This information helps development teams to stay on top of challenges, such as:
Redundant pulls and misconfigured repositories: Thesecan quickly and quietly drive up technical expenses while falling out of scope of the most relevant or critical use cases. Docker Hub’s Usage dashboards can help development teams identify patterns and optimize accordingly. They let you view usage trends across IPs and users as well, which helps with pinpointing high consumption areas and ensuring accountability in an organization when it comes to resource management.
Poor caching management: Repository insights and image tagging helps customers assess internal usage patterns, such as frequently accessed images, where there might be an opportunity to improve caching. With proper governance models, organizations can also establish policies and processes that reduce the variability of resource usage as a whole. This goal goes beyond keeping track of seasonality usage patterns to help you design more predictable usage patterns so you can budget accordingly.
Accidental automation: Accidental automated system activities can really hurt your usage. Let’s say you are using a CI/CD pipeline or automated scripts configured to pull images more often than they should. They may pull on every build instead of the actual version change, for example.
Usage dashboards can help you identify these inefficiencies by showing detailed pull data associated with automated tooling. This information can help your teams quickly identify and adjust misconfigured systems, fine-tune automations to only pull when needed, and ultimately focus on the most relevant use cases for your organization, avoiding accidental overuse of resources:
Figure 2: Details from the Usage dashboards.
Docker Hub’s Usage dashboards offer a comprehensive view of your usage data, including downloadable CSV reports that include metrics such as pull counts, repository names, IP addresses, and version checks (Figure 2). This granular approach allows your organization to gain valuable insights and trend data to help optimize your team’s workflows and inform policies.
Integrate robust operational principles into your development pipeline by leveraging these data-driven reports and maintain control over resource consumption and operational efficiency with Docker Hub.
With many organizations moving to container-based workflows, keeping track of the different versions of your images can become a problem. Even smaller organizations can have hundreds of container images spanning from one-off development tests, through emergency variants to fix problems, all the way to core production images. This leads us to the question: How can we tame our image sprawl while still rapidly iterating our images?
A common misconception is that by using the “latest” tag, you are guaranteeing that you are pulling the “latest” version of the image. Unfortunately, this assumption is wrong — all latest means is “the last image pushed to this registry.”
Read on to learn more about how to avoid this pitfall when using Docker and how to get a handle on your Docker images.
Using tags
One way to address this issue is to use tags when creating an image. Adding one or more tags to an image helps you remember what it is intended for and helps others as well. One approach is always to tag images with their semantic versioning (semver), which lets you know what version you are deploying. This sounds like a great approach, and, to some extent, it is, but there is a wrinkle.
Unless you’ve configured your registry for immutable tags, tags can be changed. For example, you could tag my-great-app as v1.0.0 and push the image to the registry. However, nothing stops your colleague from pushing their updated version of the app with tag v1.0.0 as well. Now that tag points to their image, not yours. If you add in the convenience tag latest, things get a bit more murky.
Let’s look at an example:
FROM busybox:stable-glibc
# Create a script that outputs the version
RUN echo -e "#!/bin/sh\n" > /test.sh && \
echo "echo \"This is version 1.0.0\"" >> /test.sh && \
chmod +x /test.sh
# Set the entrypoint to run the script
ENTRYPOINT ["/bin/sh", "/test.sh"]
We build the above with docker build -t tagexample:1.0.0 . and run it.
$ docker run --rm tagexample:1.0.0
This is version 1.0.0
What if we run it without a tag specified?
$ docker run --rm tagexample
Unable to find image 'tagexample:latest' locally
docker: Error response from daemon: pull access denied for tagexample, repository does not exist or may require 'docker login'.
See 'docker run --help'.
Now we build with docker build . without specifying a tag and run it.
$ docker run --rm tagexample
This is version 1.0.0
The latest tag is always applied to the most recent push that did not specify a tag. So, in our first test, we had one image in the repository with a tag of 1.0.0, but because we did not have any pushes without a tag, the latest tag did not point to an image. However, once we push an image without a tag, the latest tag is automatically applied to it.
Although it is tempting to always pull the latest tag, it’s rarely a good idea. The logical assumption — that this points to the most recent version of the image — is flawed. For example, another developer can update the application to version 1.0.1, build it with the tag 1.0.1, and push it. This results in the following:
$ docker run --rm tagexample:1.0.1
This is version 1.0.1
$ docker run --rm tagexample:latest
This is version 1.0.0
If you made the assumption that latest pointed to the highest version, you’d now be running an out-of-date version of the image.
The other issue is that there is no mechanism in place to prevent someone from inadvertently pushing with the wrong tag. For example, we could create another update to our code bringing it up to 1.0.2. We update the code, build the image, and push it — but we forget to change the tag to reflect the new version. Although it’s a small oversight, this action results in the following:
$ docker run --rm tagexample:1.0.1
This is version 1.0.2
Unfortunately, this happens all too frequently.
Using labels
Because we can’t trust tags, how should we ensure that we are able to identify our images? This is where the concept of adding metadata to our images becomes important.
The first attempt at using metadata to help manage images was the MAINTAINER instruction. This instruction sets the “Author” field (org.opencontainers.image.authors) in the generated image. However, this instruction has been deprecated in favor of the more powerful LABEL instruction. Unlike MAINTAINER, the LABEL instruction allows you to set arbitrary key/value pairs that can then be read with docker inspect as well as other tooling.
Unlike with tags, labels become part of the image, and when implemented properly, can provide a much better way to determine the version of an image. To return to our example above, let’s see how the use of a label would have made a difference.
To do this, we add the LABEL instruction to the Dockerfile, along with the key version and value 1.0.2.
FROM busybox:stable-glibc
LABEL version="1.0.2"
# Create a script that outputs the version
RUN echo -e "#!/bin/sh\n" > /test.sh && \
echo "echo \"This is version 1.0.2\"" >> /test.sh && \
chmod +x /test.sh
# Set the entrypoint to run the script
ENTRYPOINT ["/bin/sh", "/test.sh"]
Now, even if we make the same mistake above where we mistakenly tag the image as version 1.0.1, we have a way to check that does not involve running the container to see which version we are using.
Although you can use any key/value as a LABEL, there are recommendations. The OCI provides a set of suggested labels within the org.opencontainers.image namespace, as shown in the following table:
Label
Content
org.opencontainers.image.created
The date and time on which the image was built (string, RFC 3339 date-time).
org.opencontainers.image.authors
Contact details of the people or organization responsible for the image (freeform string).
org.opencontainers.image.url
URL to find more information on the image (string).
org.opencontainers.image.documentation
URL to get documentation on the image (string).
org.opencontainers.image.source
URL to the source code for building the image (string).
org.opencontainers.image.version
Version of the packaged software (string).
org.opencontainers.image.revision
Source control revision identifier for the image (string).
org.opencontainers.image.vendor
Name of the distributing entity, organization, or individual (string).
org.opencontainers.image.licenses
License(s) under which contained software is distributed (string, SPDX License List).
org.opencontainers.image.ref.name
Name of the reference for a target (string).
org.opencontainers.image.title
Human-readable title of the image (string).
org.opencontainers.image.description
Human-readable description of the software packaged in the image (string).
Because LABEL takes any key/value, it is also possible to create custom labels. For example, labels specific to a team within a company could use the com.myorg.myteam namespace. Isolating these to a specific namespace ensures that they can easily be related back to the team that created the label.
Final thoughts
Image sprawl is a real problem for organizations, and, if not addressed, it can lead to confusion, rework, and potential production problems. By using tags and labels in a consistent manner, it is possible to eliminate these issues and provide a well-documented set of images that make work easier and not harder.
This guest post was contributed by Diana Esteves, Solutions Architect, Pulumi.
Pulumi is an Infrastructure as Code (IaC) platform that simplifies resource management across any cloud or SaaS provider, including Docker. Pulumi providers are integrations with useful tools and vendors. Pulumi’s new Docker Build provider is about making your builds even easier, faster, and more reliable.
In this post, we will dive into how Pulumi’s new Docker Build provider works with Docker Build Cloud to streamline building, deploying, and managing containerized applications. First, we’ll set up a project using Docker Build Cloud and Pulumi. Then, we’ll explore cool use cases that showcase how you can leverage this provider to simplify your build and deployment pipelines.
Pulumi Docker Build provider features
Top features of the Pulumi Docker Build provider include the following:
Docker Build Cloud support: Offload your builds to the cloud and free up your local resources. Faster builds mean fewer headaches.
Multi-platform support: Build Docker images that work on different hardware architectures without breaking a sweat.
Advanced caching: Say goodbye to redundant builds. In addition to the shared caching available when you use Docker Build Cloud, this provider supports multiple cache backends, like Amazon S3, GitHub Actions, and even local disk, to keep your builds efficient.
Flexible export options: Customize where your Docker images go after they’re built — export to registries, filesystems, or wherever your workflow needs.
Getting started with Docker Build Cloud and Pulumi
Docker Build Cloud is Docker’s newest offering that provides a pair of AMD and Arm builders in the cloud and shared cache for your team, resulting in up to 39x faster image builds. Docker Personal, Pro, Team, and Business plans include a set number of Build Cloud minutes, or you can purchase a Build Cloud Team plan to add minutes. Learn more about Docker Build Cloud plans.
The example builds an NGINX Dockerfile using a Docker Build Cloud builder. We will create a Docker Build Cloud builder, create a Pulumi program in Typescript, and build our image.
Building images locally means being subject to local compute and storage availability. Pulumi allows users to build images with Docker Build Cloud.
The Pulumi Docker Build provider fully supports Docker Build Cloud, which unlocks new capabilities, as individual team members or a CI/CD pipeline can fully take advantage of improved build speeds, shared build cache, and native multi-platform builds.
If you still need to create a builder, follow the steps below; otherwise, skip to step 1C.
B. Create a new cloud builder named my-cool-builder.
Figure 1: Create the new cloud builder and call it my-cool-builder.
C. In your local machine, sign in to your Docker account.
$ docker login
D. Add your existing cloud builder endpoint.
$ docker buildx create --driver cloud ORG/BUILDER_NAME
# Replace ORG with the Docker Hub namespace of your Docker organization.
# This creates a builder named cloud-ORG-BUILDER_NAME.
# Example:
$ docker buildx create --driver cloud pulumi/my-cool-builder
# cloud-pulumi-my-cool-builder
# check your new builder is configured
$ docker buildx ls
E. Optionally, see that your new builder is available in Docker Desktop.
Figure 2: The Builders view in the Docker Desktop settings lists all available local and Docker Build Cloud builders available to the logged-in account.
For additional guidance on setting up Docker Build Cloud, refer to the Docker docs.
Step 2: Set up your Pulumi project
To create your first Pulumi project, start with a Pulumi template. Pulumi has curated hundreds of templates that are directly integrated with the Pulumi CLI via pulumi new. In particular, the Pulumi team has created a Pulumi template for Docker Build Cloud to get you started.
The Pulumi programming model centers around defining infrastructure using popular programming languages. This approach allows you to leverage existing programming tools and define cloud resources using familiar syntaxes such as loops and conditionals.
To copy the Pulumi template locally:
$ pulumi new https://github.com/pulumi/examples/tree/master/dockerbuildcloud-ts --dir hello-dbc
# project name: hello-dbc
# project description: (default)
# stack name: dev
# Note: Update the builder value to match yours
# builder: cloud-pulumi-my-cool-builder
$ cd hello-dbc
# update all npm packages (recommended)
$ npm update --save
Optionally, explore your Pulumi program. The hello-dbc folder has everything you need to build a Dockerfile into an image with Pulumi. Your Pulumi program starts with an entry point, typically a function written in your chosen programming language. This function defines the infrastructure resources and configurations for your project. For TypeScript, that file is index.ts, and the contents are shown below:
import * as dockerBuild from "@pulumi/docker-build";
import * as pulumi from "@pulumi/pulumi";
const config = new pulumi.Config();
const builder = config.require("builder");
const image = new dockerBuild.Image("image", {
// Configures the name of your existing buildx builder to use.
// See the Pulumi.<stack>.yaml project file for the builder configuration.
builder: {
name: builder, // Example, "cloud-pulumi-my-cool-builder",
},
context: {
location: "app",
},
// Enable exec to run a custom docker-buildx binary with support
// for Docker Build Cloud (DBC).
exec: true,
push: false,
});
Step 3: Build your Docker image
Run the pulumi up command to see the image being built with the newly configured builder:
$ pulumi up --yes
You can follow the browser link to the Pulumi Cloud dashboard and navigate to the Image resource to confirm it’s properly configured by noting the builder parameter.
Figure 3: Navigate to the Image resource to check the configuration.
Optionally, also check your Docker Build Cloud dashboard for build minutes usage:
Figure 4: The build.docker.com view shows the user has selected the Cloud builders from the left menu and the product dashboard is shown on the right side.
Congratulations! You have built an NGINX Dockerfile with Docker Build Cloud and Pulumi. This was achieved by creating a new Docker Build Cloud builder and passing that to a Pulumi template. The Pulumi CLI is then used to deploy the changes.
Advanced use cases with buildx and BuildKit
To showcase popular buildx and BuildKit features, test one or more of the following Pulumi code samples. These include multi-platform, advanced caching, and exports. Note that each feature is available as an input (or parameter) in the Pulumi Docker Build Image resource.
Multi-platform image builds for Docker Build Cloud
Docker images can support multiple platforms, meaning a single image may contain variants for architectures and operating systems.
The following code snippet is analogous to invoking a build from the Docker CLI with the --platform flag to specify the target platform for the build output.
import * as dockerBuild from "@pulumi/docker-build";
const image = new dockerBuild.Image("image", {
// Build a multi-platform image manifest for ARM and AMD.
platforms: [
dockerBuild.Platform.Linux_amd64,
dockerBuild.Platform.Linux_arm64,
],
push: false,
});
Deploy the changes made to the Pulumi program:
$ pulumi up --yes
Caching from and to AWS ECR
Maintaining cached layers while building Docker images saves precious time by enabling faster builds. However, utilizing cached layers has been historically challenging in CI/CD pipelines due to recycled environments between builds. The cacheFrom and cacheTo parameters allow programmatic builds to optimize caching behavior.
Update your Docker image resource to take advantage of caching:
import * as dockerBuild from "@pulumi/docker-build";
import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws"; // Required for ECR
// Create an ECR repository for pushing.
const ecrRepository = new aws.ecr.Repository("ecr-repository", {});
// Grab auth credentials for ECR.
const authToken = aws.ecr.getAuthorizationTokenOutput({
registryId: ecrRepository.registryId,
});
const image = new dockerBuild.Image("image", {
push: true,
// Use the pushed image as a cache source.
cacheFrom: [{
registry: {
ref: pulumi.interpolate`${ecrRepository.repositoryUrl}:cache`,
},
}],
cacheTo: [{
registry: {
imageManifest: true,
ociMediaTypes: true,
ref: pulumi.interpolate`${ecrRepository.repositoryUrl}:cache`,
},
}],
// Provide our ECR credentials.
registries: [{
address: ecrRepository.repositoryUrl,
password: authToken.password,
username: authToken.userName,
}],
})
Notice the declaration of additional resources for AWS ECR.
Export builds as a tar file
Exporting allows us to share or back up the resulting image from a build invocation.
To export the build as a local .tar file, modify your resource to include the exports Input:
Review the Pulumi Docker Build provider guide to explore other Docker Build features, such as build arguments, build contexts, remote contexts, and more.
Next steps
Infrastructure as Code (IaC) is key to managing modern cloud-native development, and Docker lets developers create and control images with Dockerfiles and Docker Compose files. But when the situation gets more complex, like deploying across different cloud platforms, Pulumi can offer additional flexibility and advanced infrastructure features. The Docker Build provider supports Docker Build Cloud, streamlining building, deploying, and managing containerized applications, which helps development teams work together more effectively and maintain agility.
Pulumi’s latest Docker Build provider, powered by BuildKit, improves flexibility and efficiency in Docker builds. By applying IaC principles, developers manage infrastructure with code, even in intricate scenarios. This means you can focus on building and deploying your containerized workloads without the hassle of complex infrastructure challenges.
In this fourth installment of our Crossplane tutorial series, we are exploring Configuration Packages. They allow us to package Crossplane Compositions as OCI (Docker) images and distribute them to control plane clusters.
▬▬▬▬▬▬ Sponsorships ▬▬▬▬▬▬ If you are interested in sponsoring this channel, please use https://calendly.com/vfarcic/meet to book a timeslot that suits and we’ll go over the details. Or feel free to contact me over Twitter or LinkedIn (see below)
What do Docker and IKEA Retail have in common? Both companies have changed how products are built, stored, and shipped. In IKEA Retail’s case, they created the market of flat-packed furniture items, which made everything from shipping, warehousing, and delivering their furniture to the end location much easier and more cost effective. This parallels what Docker has done for developers. Docker has changed the way that software is built, shipped, and stored, with Docker Images taking up much less space “shelf” space.
In this post, contributing authors Karan Honavar and Fernando Dorado Rueda from IKEA Retail walk through their MLOps solution, built with Docker.
Machine learning (ML) deployment, the act of shifting an ML model from the developmental stage to a live production environment, is paramount to translating complex algorithms into real-world solutions. Yet, this intricate process isn’t without its challenges, including:
Complexity and opacity: With ML models often veiled in complexity, deciphering their logic can be taxing. This obscurity impedes trust and complicates the explanation of decisions to stakeholders.
Adaptation to changing data patterns: The shifting landscape of real-world data can deviate from training sets, causing “concept drift.” Addressing this requires vigilant retraining, an arduous task that wastes time and resources.
Real-time data processing: Handling the deluge of data necessary for accurate predictions can burden systems and impede scalability.
Varied deployment methods: Whether deployed locally, in the cloud, or via web services, each method brings unique challenges, adding layers of complexity to an already intricate procedure.
Security and compliance: Ensuring that ML models align with rigorous regulations, particularly around private information, necessitates a focus on lawful implementation.
Ongoing maintenance and monitoring: The journey doesn’t end with deployment. Constant monitoring is vital to sustain the model’s health and address emerging concerns.
These factors represent substantial obstacles, but they are not insurmountable. We can streamline the journey from the laboratory to the real world by standardizing Docker images for efficient ML model deployment.
This article will delve into the creation, measurement, deployment, and interaction with Dockerized ML models. We will demystify the complexities and demonstrate how Docker can catalyze cutting-edge concepts into tangible benefits.
Standardization deployment process via Docker
In the dynamic realm of today’s data-driven enterprises, such as our case at IKEA Retail, the multitude of tools and deployment strategies serves both as a boon and a burden. Innovation thrives, but so too does complexity, giving rise to inconsistency and delays. The antidote? Standardization. It’s more than just a buzzword; it’s a method to pave the way to efficiency, compliance, and seamless integration.
Enter Docker, the unsung hero in this narrative. In the evolving field of ML deployment, Docker offers agility and uniformity. It has reshaped the landscape by offering a consistent environment from development to production. The beauty of Docker lies in its containerization technology, enabling developers to wrap up an application with all the parts it needs, such as libraries and other dependencies, and ship it all out as one package.
At IKEA Retail, diverse teams — including hybrid data scientist teams and R&D units — conceptualize and develop models, each selecting drivers and packaging libraries according to their preferences and requirements. Although virtual environments provide a certain level of support, they can also present compatibility challenges when transitioning to a production environment.
This is where Docker becomes an essential tool in our daily operations, offering simplification and a marked acceleration in the development and deployment process. Here are key advantages:
Portability: With Docker, the friction between different computing environments melts away. A container runs uniformly, regardless of where it’s deployed, bringing robustness to the entire pipeline.
Efficiency: Docker’s lightweight nature ensures that resources are optimally utilized, thereby reducing overheads and boosting performance.
Scalability: With Docker, scaling your application or ML models horizontally becomes a breeze. It aligns perfectly with the orchestrated symphony that large-scale deployment demands.
Then, there’s Seldon-Core, a solution chosen by IKEA Retail’s forward-thinking MLOps (machine learning operations) team. Why? Because it transforms ML models into production-ready microservices, regardless of the model’s origin (TensorFlow, PyTorch, H2O, etc.) or language (Python, Java, etc.). But that’s not all. Seldon-Core scales precisely, enabling everything from advanced metrics and logging to explainers and A/B testing.
This combination of Docker and Seldon-Core forms the heart of our exploration today. Together, they sketch the blueprint for a revolution in ML deployment. This synergy is no mere technical alliance; it’s a transformative collaboration that redefines deploying, monitoring, and interacting with ML models.
Through the looking glass of IKEA Retail’s experience, we’ll unearth how this robust duo — Docker and Seldon-Core — can turn a convoluted task into a streamlined, agile operation and how you can harness real-time metrics for profound insights.
Dive into this new MLOps era with us. Unlock efficiency, scalability, and a strategic advantage in ML production. Your innovation journey begins here, with Docker and Seldon-Core leading the way. This is more than a solution; it’s a paradigm shift.
In the rest of this article, we will cover deployment steps, including model preparation, encapsulating the model into an Docker image, and testing. Let’s get started.
Prerequisites
The following items must be present to replicate this example:
Docker: Ensure Docker is up and running, easily achievable through solutions like Docker Desktop
Python: Have a local installation at the ready (+3.7)
Model preparation
Model training and simple evaluation
Embarking on the journey to deploying an ML model is much like crafting a masterpiece: The canvas must be prepared, and every brushstroke must be deliberate. However, the focus of this exploration isn’t the art itself but rather the frame that holds it — the standardization of ML models, regardless of their creation or the frameworks used.
The primary objective of this demonstration is not to augment the model’s performance but rather to elucidate the seamless transition from local development to production deployment. It is imperative to note that the methodology we present is universally applicable across different models and frameworks. Therefore, we have chosen a straightforward model as a representative example. This choice is intentional, allowing readers to concentrate on the underlying process flows, which can be readily adapted to more sophisticated models that may require refined hyperparameter tuning and meticulous model selection.
By focusing on these foundational principles, we aim to provide a versatile and accessible guide that transcends the specificities of individual models or use cases. Let’s delve into this process.
To align with our ethos of transparency and consumer privacy and to facilitate your engagement with this approach, a public dataset is employed for a binary classification task.
In the following code excerpt, you’ll find the essence of our training approach, reflecting how we transform raw data into a model ready for real-world challenges:
import os
import pickle
import numpy as np
import pandas as pd
from sklearn import datasets
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression, Perceptron
from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score
# Load the breast cancer dataset
X, y = datasets.load_breast_cancer(return_X_y=True)
# Split the dataset into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.9, random_state=0)
# Combine X_test and y_test into a single DataFrame
X_test_df = pd.DataFrame(X_test, columns=[f"feature_{i}" for i in range(X_test.shape[1])])
y_test_df = pd.DataFrame(y_test, columns=["target"])
df_test = pd.concat([X_test_df, y_test_df], axis=1)
# Define the path to store models
model_path = "models/"
# Create the folder if it doesn't exist
if not os.path.exists(model_path):
os.makedirs(model_path)
# Define a list of classifier parameters
parameters = [
{"clf": LogisticRegression(solver="liblinear", multi_class="ovr"), "name": f"{model_path}/binary-lr.joblib"},
{"clf": Perceptron(eta0=0.1, random_state=0), "name": f"{model_path}/binary-percept.joblib"},
]
# Iterate through each parameter configuration
for param in parameters:
clf = param["clf"] # Retrieve the classifier from the parameter dictionary
clf.fit(X_train, y_train) # Fit the classifier on the training data
# Save the trained model to a file using pickle
model_filename = f"{param['name']}"
with open(model_filename, 'wb') as model_file:
pickle.dump(clf, model_file)
print(f"Model saved in {model_filename}")
# Simple Model Evaluation
model_path = 'models/binary-lr.joblib'
with open(model_path, 'rb') as model_file:
loaded_model = pickle.load(model_file)
# Make predictions using the loaded model
predictions = loaded_model.predict(X_test)
# Calculate metrics (accuracy, precision, recall, f1-score)
accuracy = accuracy_score(y_test, predictions)
precision = precision_score(y_test, predictions)
recall = recall_score(y_test, predictions)
f1 = f1_score(y_test, predictions)
Model class creation
With the model files primed, the task at hand shifts to the crafting of the model class — an essential architectural element that will later reside within the Docker image. Like a skilled sculptor, we must shape this class, adhering to the exacting standards proposed by Seldon:
import joblib
import logging
class Score:
"""
Class to hold metrics for binary classification, including true positives (TP), false positives (FP),
true negatives (TN), and false negatives (FN).
"""
def __init__(self, TP=0, FP=0, TN=0, FN=0):
self.TP = TP # True Positives
self.FP = FP # False Positives
self.TN = TN # True Negatives
self.FN = FN # False Negatives
class DockerModel:
"""
Class for loading and predicting using a pre-trained model, handling feedback to update metrics,
and providing those metrics.
"""
result = {} # Dictionary to store input data
def __init__(self, model_name="models/binary-lr.joblib"):
"""
Initialize DockerModel with metrics and model name.
:param model_name: Path to the pre-trained model.
"""
self.scores = Score(0, 0, 0, 0)
self.loaded = False
self.model_name = model_name
def load(self):
"""
Load the model from the provided path.
"""
self.model = joblib.load(self.model_name)
logging.info(f"Model {self.model_name} Loaded")
def predict(self, X, features_names=None, meta=None):
"""
Predict the target using the loaded model.
:param X: Features for prediction.
:param features_names: Names of the features, optional.
:param meta: Additional metadata, optional.
:return: Predicted target values.
"""
self.result['shape_input_data'] = str(X.shape)
logging.info(f"Received request: {X}")
if not self.loaded:
self.load()
self.loaded = True
predictions = self.model.predict(X)
return predictions
def send_feedback(self, features, feature_names, reward, truth, routing=""):
"""
Provide feedback on predictions and update the metrics.
:param features: Features used for prediction.
:param feature_names: Names of the features.
:param reward: Reward signal, not used in this context.
:param truth: Ground truth target values.
:param routing: Routing information, optional.
:return: Empty list as return value is not used.
"""
predicted = self.predict(features)
logging.info(f"Predicted: {predicted[0]}, Truth: {truth[0]}")
if int(truth[0]) == 1:
if int(predicted[0]) == int(truth[0]):
self.scores.TP += 1
else:
self.scores.FN += 1
else:
if int(predicted[0]) == int(truth[0]):
self.scores.TN += 1
else:
self.scores.FP += 1
return [] # Ignore return statement as its not used
def calculate_metrics(self):
"""
Calculate the accuracy, precision, recall, and F1-score.
:return: accuracy, precision, recall, f1_score
"""
total_samples = self.scores.TP + self.scores.TN + self.scores.FP + self.scores.FN
# Check if there are any samples to avoid division by zero
if total_samples == 0:
logging.warning("No samples available to calculate metrics.")
return 0, 0, 0, 0 # Return zeros for all metrics if no samples
accuracy = (self.scores.TP + self.scores.TN) / total_samples
# Check if there are any positive predictions to calculate precision
positive_predictions = self.scores.TP + self.scores.FP
precision = self.scores.TP / positive_predictions if positive_predictions != 0 else 0
# Check if there are any actual positives to calculate recall
actual_positives = self.scores.TP + self.scores.FN
recall = self.scores.TP / actual_positives if actual_positives != 0 else 0
# Check if precision and recall are non-zero to calculate F1-score
if precision + recall == 0:
f1_score = 0
else:
f1_score = 2 * (precision * recall) / (precision + recall)
# Return the calculated metrics
return accuracy, precision, recall, f1_score
def metrics(self):
"""
Generate metrics for monitoring.
:return: List of dictionaries containing accuracy, precision, recall, and f1_score.
"""
accuracy, precision, recall, f1_score = self.calculate_metrics()
return [
{"type": "GAUGE", "key": "accuracy", "value": accuracy},
{"type": "GAUGE", "key": "precision", "value": precision},
{"type": "GAUGE", "key": "recall", "value": recall},
{"type": "GAUGE", "key": "f1_score", "value": f1_score},
]
def tags(self):
"""
Retrieve metadata when generating predictions
:return: Dictionary the intermediate information
"""
return self.result
Let’s delve into the details of the functions and classes within the DockerModel class that encapsulates these four essential aspects:
Loading and predicting:
load(): This function is responsible for importing the pretrained model from the provided path. It’s usually called internally before making predictions to ensure the model is available.
predict(X, features_names=None, meta=None): This function deploys the loaded model to make predictions. It takes in the input features X, optional features_names, and optional metadata meta, returning the predicted target values.
Feedback handling:
send_feedback(features, feature_names, reward, truth, routing=""): This function is vital in adapting the model to real-world feedback. It accepts the input data, truth values, and other parameters to assess the model’s performance. The feedback updates the model’s understanding, and the metrics are calculated and stored for real-time analysis. This facilitates continuous retraining of the model.
Metrics calculation:
calculate_metrics(): This function calculates the essential metrics of accuracy, precision, recall, and F1-score. These metrics provide quantitative insights into the model’s performance, enabling constant monitoring and potential improvement.
Score class: This auxiliary class is used within the DockerModel to hold metrics for binary classification, including true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN). It helps keep track of these parameters, which are vital for calculating the aforementioned metrics.
Monitoring assistance:
metrics(): This function generates the metrics for model monitoring. It returns a list of dictionaries containing the calculated accuracy, precision, recall, and F1 score. These metrics are compliant with Prometheus Metrics, facilitating real-time monitoring and analysis.
tags(): This function is designed to retrieve custom metadata data when generating predictions, aiding in monitoring and debugging. It returns a dictionary, which can help track and understand the nature of the requests.
Together, these functions and classes form a cohesive and robust structure that supports the entire lifecycle of an ML model. From the moment of inception (loading and predicting) through its growth (feedback handling) and evaluation (metrics calculation), to its continuous vigilance (monitoring assistance), the architecture is designed to standardize and streamline the process of deploying and maintaining ML models in a real-world production environment.
This model class is more than code; it’s the vessel that carries our ML model from a local environment to the vast sea of production. It’s the vehicle for standardization, unlocking efficiency and consistency in deploying models.
At this stage, we’ve prepared the canvas and outlined the masterpiece. Now, it’s time to dive deeper and explore how this model is encapsulated into a Docker image, an adventure that blends technology and strategy to redefine ML deployment.
Testing model locally
Before venturing into creating a Docker image, testing the model locally is vital. This step acts as a rehearsal before the main event, providing a chance to ensure that the model is performing as expected with the testing data.
The importance of local testing lies in its ability to catch issues early, avoiding potential complications later in the deployment process. Following the example code provided below, it can confirm that the model is ready for its next phase if it provides the expected prediction in the expected format:
from DockerModel import DockerModel
demoModel = DockerModel()
demoModel.predict(X_test) # Can take the entire testing dataset or individual predictions
The expected output should match the format of the class labels you anticipate from the model. If everything works correctly, you’re assured that the model is well prepared for the next grand step: encapsulation within a Docker image.
Local testing is more than a technical process; it’s a quality assurance measure that stands as a gatekeeper, ensuring that only a well-prepared model moves forward. It illustrates the meticulous care taken in the deployment process, reflecting a commitment to excellence that transcends code and resonates with the core values of standardization and efficiency.
With the local testing accomplished, we stand on the threshold of a new frontier: creating the Docker image. Let’s continue this exciting journey, knowing each step is a stride toward innovation and mastery in ML deployment.
Encapsulating the model into a Docker image
In our IKEA Retail MLOps view, a model is not simply a collection of code. Rather, it is a sophisticated assembly comprising code, dependencies, and ML artifacts, all encapsulated within a versioned and registered Docker image. This composition is carefully designed, reflecting the meticulous planning of the physical infrastructure.
What is Docker’s role in MLOps?
Docker plays a vital role in MLOps, providing a standardized environment that streamlines the transition from development to production:
Streamlining deployment: Docker containers encapsulate everything an ML model needs to run, easing the deployment process.
Facilitating collaboration: Using Docker, data scientists and engineers can ensure that models and their dependencies remain consistent across different stages of development.
Enhancing model reproducibility: Docker provides a uniform environment that enhances the reproducibility of models, a critical aspect in machine learning.
Integrating with orchestration tools: Docker can be used with orchestration platforms like Kubernetes, enabling automated deployment, scaling, and management of containerized applications.
Docker and containerization are more than technology tools; they catalyze innovation and efficiency in MLOps. Ensuring consistency, scalability and agility, Docker unlocks new potential and opens the way for a more agile and robust ML deployment process. Whether you are a developer, a data scientist, or an IT professional, understanding Docker is critical to navigating the complex and multifaceted landscape of modern data-driven applications.
Dockerfile creation
Creating a Dockerfile is like sketching the architectural plan of a building. It outlines the instructions for creating a Docker image to run the application in a coherent, isolated environment. This design ensures that the entire model — including its code, dependencies, and unique ML artifacts — is treated as a cohesive entity, aligning with the overarching vision of IKEA Retail’s MLOps approach.
In our case, we have created a Dockerfile with the express purpose of encapsulating not only the code but all the corresponding artifacts of the model. This deliberate design facilitates a smooth transition to production, effectively bridging the gap between development and deployment.
We used the following Dockerfile for this demonstration, which represents a tangible example of how IKEA Retail’s MLOps approach is achieved through thoughtful engineering and strategic implementation.
# Use an official Python runtime as a parent image.
# Using a slim image for a smaller final size and reduced attack surface.
FROM python:3.9-slim
# Set the maintainer label for metadata.
LABEL maintainer="fernandodorado.rueda@ingka.com"
# Set environment variables for a consistent build behavior.
# Disabling the buffer helps to log messages synchronously.
ENV PYTHONUNBUFFERED=1
# Set a working directory inside the container to store all our project files.
WORKDIR /app
# First, copy the requirements file to leverage Docker's cache for dependencies.
# By doing this first, changes to the code will not invalidate the cached dependencies.
COPY requirements.txt requirements.txt
# Install the required packages listed in the requirements file.
# It's a good practice to include the --no-cache-dir flag to prevent the caching of dependencies
# that aren't necessary for executing the application.
RUN pip install --no-cache-dir -r requirements.txt
# Copy the rest of the code and model files into the image.
COPY DockerModel.py DockerModel.py
COPY models/ models/
# Expose ports that the application will run on.
# Port 5000 for GRPC
# Port 9000 for REST
EXPOSE 5000 9000
# Set environment variables used by the application.
ENV MODEL_NAME DockerModel
ENV SERVICE_TYPE MODEL
# Change the owner of the directory to user 8888 for security purposes.
# It can prevent unauthorised write access by the application itself.
# Make sure to run the application as this non-root user later if applicable.
RUN chown -R 8888 /app
# Use the exec form of CMD so that the application you run will receive UNIX signals.
# This is helpful for graceful shutdown.
# Here we're using seldon-core-microservice to serve the model.
CMD exec seldon-core-microservice $MODEL_NAME --service-type $SERVICE_TYPE
This Dockerfile contains different parts:
FROM python:3.9-slim: This line chooses the official Python 3.9 slim image as the parent image. It is favored for its reduced size and attack surface, enhancing both efficiency and security.
LABEL maintainer="fernandodorado.rueda@ingka.com": A metadata label that specifies the maintainer of the image, providing contact information.
ENV PYTHONUNBUFFERED=1: Disabling Python’s output buffering ensures that log messages are emitted synchronously, aiding in debugging and log analysis.
WORKDIR /app: Sets the working directory inside the container to /app, a centralized location for all project files.
COPY requirements.txt requirements.txt: Copies the requirements file into the image. Doing this before copying the rest of the code leverages Docker’s caching mechanism, making future builds faster. This file must contain the “seldon-core” package:
RUN pip install --no-cache-dir -r requirements.txt: Installs required packages as listed in the requirements file. The flag -no-cache-dir prevents unnecessary caching of dependencies, reducing the image size.
COPY DockerModel.py DockerModel.py: Copies the main Python file into the image.
COPY models/ models/: Copies the model files into the image.
EXPOSE 5000 9000: Exposes ports 5000 (GRPC) and 9000 (REST), allowing communication with the application inside the container.
ENV MODEL_NAME DockerModel: Sets the environment variable for the model name.
ENV SERVICE_TYPE MODEL: Sets the environment variable for the service type.
RUN chown -R 8888 /app: Changes the owner of the directory to user 8888. Running the application as a non-root user helps mitigate the risk of unauthorized write access.
CMD exec seldon-core-microservice $MODEL_NAME --service-type $SERVICE_TYPE: Executes the command to start the service using seldon-core-microservice. It also includes the model name and service type as parameters. Using exec ensures the application receives UNIX signals, facilitating graceful shutdown.
Building and pushing Docker image
1. Installing Docker Desktop
If not already installed, Docker Desktop is recommended for this task. Docker Desktop provides a graphical user interface that simplifies the process of building, running, and managingDocker containers. Docker Desktop also supports Kubernetes, offering an easy way to create a local cluster.
2. Navigating to the Project directory
Open a terminal or command prompt.
Navigate to the folder where the Dockerfile and other necessary files are located.
3. Building the Image
Execute the command: docker build . -t docker-model:1.0.0.
docker build . instructs Docker to build the image using the current directory (.).
-t docker-model:1.0.0 assigns a name (docker-model) and tag (1.0.0) to the image.
The build process will follow the instructions defined in the Dockerfile, creating a Docker image encapsulating the entire environment needed to run the model.
4. Pushing the image
If needed, the image can be pushed to a container registry like Docker Hub, or a private registry within an organization.
For this demonstration, the image is being kept in the local container registry, simplifying the process and removing the need for authentication with an external registry.
Deploy ML model using Docker: Unleash it into the world
Once the Docker image is built, running it is relatively straightforward. Let’s break down this process:
docker run --rm --name docker-model -p 9000:9000 docker-model:1.0.0
Components of the command:
docker run: This is the base command to run a Docker container.
-rm: This flag ensures that the Docker container is automatically removed once it’s stopped. It helps keep the environment clean, especially when you run containers for testing or short-lived tasks.
-name docker-model: Assigns a name to the running container.
p 9000:9000: This maps port 9000 on the host machine to port 9000 on the Docker container. The format is p <host_port>:<container_port>. Because the Dockerfile mentions that the application will be exposing ports 5000 for GRPC and 9000 for REST, this command makes sure the REST endpoint is available to external users or applications through port 9000 on the host.
docker-model:1.0.0: This specifies the name and tag of the Docker image to run. docker-model is the name, and 1.0.0 is the version tag we assigned during the build process.
What happens next
On executing the command, Docker will initiate a container instance from the docker-model:1.0.0 image.
The application within the Docker container will start and begin listening for requests on port 9000 (as specified).
With the port mapping, any incoming requests on port 9000 of the host machine will be forwarded to port 9000 of the Docker container.
The application can now be accessed and interacted with as if it were running natively on the host machine.
Test deployed model using Docker
With the Docker image in place, it’s time to see the model in action.
Generate predictions
The path from model to prediction is a delicate process, requiring an understanding of the specific input-output type that Seldon accommodates (e.g., ndarray, JSON data, STRDATA).
In our scenario, the model anticipates an array, and thus, the key in our payload is “ndarray.” Here’s how we orchestrate this:
import requests
import json
URL = "http://localhost:9000/api/v1.0/predictions"
def send_prediction_request(data):
# Create the headers for the request
headers = {'Content-Type': 'application/json'}
try:
# Send the POST request
response = requests.post(URL, headers=headers, json=data)
# Check if the request was successful
response.raise_for_status() # Will raise HTTPError if the HTTP request returned an unsuccessful status code
# If successful, return the JSON data
return response.json()
except requests.ConnectionError:
raise Exception("Failed to connect to the server. Is it running?")
except requests.Timeout:
raise Exception("Request timed out. Please try again later.")
except requests.RequestException as err:
# For any other requests exceptions, re-raise it
raise Exception(f"An error occurred with your request: {err}")
X_test
# Define the data payload (We can also use X_test[0:1].tolist() instead of the raw array)
data_payload = {
"data": {
"ndarray": [
[
1.340e+01, 2.052e+01, 8.864e+01, 5.567e+02, 1.106e-01, 1.469e-01,
1.445e-01, 8.172e-02, 2.116e-01, 7.325e-02, 3.906e-01, 9.306e-01,
3.093e+00, 3.367e+01, 5.414e-03, 2.265e-02, 3.452e-02, 1.334e-02,
1.705e-02, 4.005e-03, 1.641e+01, 2.966e+01, 1.133e+02, 8.444e+02,
1.574e-01, 3.856e-01, 5.106e-01, 2.051e-01, 3.585e-01, 1.109e-01
]
]
}
}
# Get the response and print it
try:
response = send_prediction_request(data_payload)
pretty_json_response = json.dumps(response, indent=4)
print(pretty_json_response)
except Exception as err:
print(err)
The prediction of our model will be similar to this dictionary:
The response from the model will contain several keys:
"data": Provides the generated output by our model. In our case, it’s the predicted class.
"meta": Contains metadata and model metrics. It shows the actual values of the classification metrics, including accuracy, precision, recall, and f1_score.
"tags": Contains intermediate metadata. This could include anything you want to track, such as the shape of the input data.
The structure outlined above ensures that not only can we evaluate the final predictions, but we also gain insights into intermediate results. These insights can be instrumental in understanding predictions and debugging any potential issues.
This stage marks a significant milestone in our journey from training a model to deploying and testing it within a Docker container. We’ve seen how to standardize an ML model and how to set it up for real-world predictions. With this foundation, you’re well-equipped to scale, monitor, and further integrate this model into a full-fledged production environment.
Send feedback in real-time and calculate metrics
The provisioned /feedback endpoint facilitates this learning by allowing truth values to be sent back to the model once they are available. As these truth values are received, the model’s metrics are updated and can be scraped by other tools for real-time analysis and monitoring. In the following code snippet, we iterate over the test dataset and send the truth value to the /feedback endpoint, using a POST request:
import requests
import json
URL = "http://localhost:9000/api/v1.0/feedback"
def send_prediction_feedback(data):
# Create the headers for the request
headers = {'Content-Type': 'application/json'}
try:
# Send the POST request
response = requests.post(URL, headers=headers, json=data)
# Check if the request was successful
response.raise_for_status() # Will raise HTTPError if the HTTP request returned an unsuccessful status code
# If successful, return the JSON data
return response.json()
except requests.ConnectionError:
raise Exception("Failed to connect to the server. Is it running?")
except requests.Timeout:
raise Exception("Request timed out. Please try again later.")
except requests.RequestException as err:
# For any other requests exceptions, re-raise it
raise Exception(f"An error occurred with your request: {err}")
for i in range(len(X_test)):
payload = {'request': {'data': {'ndarray': [X_test[i].tolist()]}}, 'truth': {'data': {'ndarray': [int(y_test[i])]}}}
# Get the response and print it
try:
response = send_prediction_feedback(payload)
pretty_json_response = json.dumps(response, indent=4) # Pretty-print JSON
print(pretty_json_response)
except Exception as err:
print(err)
After processing the feedback, the model calculates and returns key metrics, including accuracy, precision, recall, and F1-score. These metrics are then available for analysis:
What makes this approach truly powerful is that the model’s evolution is no longer confined to the training phase. Instead, it’s in a continual state of learning, adjustment, and refinement, based on real-world feedback.
This way, we’re not just deploying a static prediction engine but fostering an evolving intelligent system that can better align itself with the changing landscape of data it interprets. It’s a holistic approach to machine learning deployment that encourages continuous improvement and real-time adaptation.
Conclusions
At IKEA Retail, Docker has become an indispensable element in our daily MLOps activities, serving as a catalyst that accelerates the development and deployment of models, especially when transitioning to production. The transformative impact of Docker unfolds through a spectrum of benefits that not only streamlines our workflow but also fortifies it:
Standardization: Docker orchestrates a consistent environment during the development and deployment of any ML model, fostering uniformity and coherence across the lifecycle.
Compatibility: With support for diverse environments and seamless multi-cloud or on-premise integration, Docker bridges gaps and ensures a harmonious workflow.
Isolation: Docker ensures that applications and resources are segregated, offering an isolated environment that prioritizes efficiency and integrity.
Security: Beyond mere isolation, Docker amplifies security by completely segregating applications from each other. This robust separation empowers us with precise control over traffic flow and management, laying a strong foundation of trust.
These attributes translate into tangible advantages in our MLOps journey, sculpting a landscape that’s not only innovative but also robust:
Agile development and deployment environment: Docker ignites a highly responsive development and deployment environment, enabling seamless creation, updating, and deployment of ML models.
Optimized resource utilization: Utilize compute/GPU resources efficiently within a shared model, maximizing performance without compromising flexibility.
Scalable deployment: Docker’s architecture allows for the scalable deployment of ML models, adapting effortlessly to growing demands.
Smooth release cycles: Integrating seamlessly with our existing CI/CD pipelines, Docker smoothens the model release cycle, ensuring a continuous flow of innovation.
Effortless integration with monitoring tools: Docker’s compatibility extends to monitoring stacks like Prometheus + Grafana, creating a cohesive ecosystem fully aligned with our MLOps approach when creating and deploying models in production.
The convergence of these benefits elevates IKEA Retail’s MLOps strategy, transforming it into a symphony of efficiency, security, and innovation. Docker is not merely a tool: Docker is a philosophy that resonates with our pursuit of excellence. Docker is the bridge that connects creativity with reality, and innovation with execution.
In the complex world of ML deployment, we’ve explored a path less trodden but profoundly rewarding. We’ve tapped into the transformative power of standardization, unlocking an agile and responsive way to deploy and engage with ML models in real-time.
But this is not a conclusion; it’s a threshold. New landscapes beckon, brimming with opportunities for growth, exploration, and innovation. The following steps will continue the current approach:
Scaling with Kubernetes: Unleash the colossal potential of Kubernetes, a beacon of flexibility and resilience, guiding you to a horizon of unbounded possibilities.
Applying real-time monitoring and alerting systems based on open source technologies, such as Prometheus and Grafana.
Connecting a data-drift detector for real-time detection: Deployment and integration of drift detectors to detect changes in data in real-time.
We hope this exploration will empower you to redefine your paths, ignite new ideas, and push the boundaries of what’s possible. The gateway to an extraordinary future is open, and the key is in our hands.
Connexion
