Outdoor light displays are a fun holiday tradition — from simple light strings hung from the eaves to elaborate scenes that bring out your competitive spirit. If using open source tools, thousands of feet of electrical cables, custom controllers, and your favorite music to build complex projects appeals to you, then the holiday season offers the perfect opportunity to indulge your creative passion.
I personally run home light shows at Halloween and Christmas that feature up to 30,000 individually addressable LED lights synchronized with dozens of different songs. It’s been an interesting learning journey over the past five years, but it is also one that almost anyone can pursue, regardless of technical ability. Read on for tips on how to make a display that’s the highlight of your neighborhood.
Getting started with outdoor light shows
As you might expect, light shows are built using a combination of hardware and software. The hardware includes the lights, props, controllers, and cabling. On the software side, there are different tools for the programming, also called sequencing, of the lights as well as the playback of the show.
Figure 1: Light show hardware includes the lights, props, controllers, and cabling.
Hardware requirements
Lights
Let’s look more closely at the hardware behind the scenes starting with the lights. Multiple types of lights can be used in displays, but I’ll keep it simple and focus on the most popular choice. Most shows are built around 12mm RGB LED lights that support the WS2811 protocol, often referred to as pixels or nodes. Generally, these are not available at retail stores. That means you’ll need to order them online, and I recommend choosing a vendor that specializes in light displays. I have purchased lights from a few different vendors, but recently I’ve been using Wally’s Lights, Visionary Light Shows, and Your Pixel Store.
Props
The lights are mounted into different props — such as a spider for Halloween or a snowflake for the winter holidays. You can either purchase these props, which are usually made out of the same plastic cardboard material used in yard signs, or you can make them yourself. Very few vendors sell pre-built props, so be ready to push the pixels by hand — yes, in my display either I or someone in my family pushed each of the 30,000 lights into place when we initially built the props. I get most of my props from EFL Designs, Gilbert Engineering, or Boscoyo Studio.
Figure 2: The lights are mounted into different props, which you can purchase or make yourself.
Controllers
Once your props are ready to go, you’ll need something to drive them. This is where controllers come in (Figure 3). Like the props and lights, you can get your controllers from various specialized vendors and, to a large extent, you can mix and match different brands in the same show because they all speak the same protocols to control the pixels (usually E1.31 or DDP).
You can purchase controllers that are ready to run, or you can buy the individual components and build your own boxes — I grew up building PCs, so I love this degree of flexibility. However, I do tend to buy pre-configured controllers, because I like having a warranty from the manufacturer. My controllers all come from HolidayCoro, but Falcon controllers are also popular.
Figure 3: Once your props are ready to go, you’ll need a controller.
The number of controllers you need depends on the number of lights in your show. Most controllers have multiple outputs, and each output can drive a certain number of lights. I typically plan for about 400 lights per output. Plus, I use about three main controllers and four receiver boxes. Note that long-range receivers are a way of extending the distance you can place lights from the main controller, but this is more of an advanced topic and not one I’ll cover in this introductory article.
Cables
Although controllers are powered by standard household outlets, the connection from the controllers to the lights happens over specialized cabling. These extension cables contain three wires. Two are used to send power to the lights (either 5v or 12v), and a third is used to send data. Basically, this third wire sends instructions like “light 1,232 turn green for .5 seconds then fade to off over .25 seconds.” You can get these extension cables from any vendor that sells pixels.
Additionally, all of the controllers need to be on the same Ethernet network. Many folks run their shows on wireless networks, but I prefer a wired setup for increased performance and reliability.
Software and music
At this point, you have a bunch of props with lights connected to networked controllers via specialized cabling. But, how do you make them dance? That’s where the software comes in.
xLights
Many hobbyists use xLights to program their lights. This software is open source and available for Mac, Windows, and Linux, and it works with three basic primitives: props, effects, and time. You can choose what effect you want to apply to a given prop at a given time (Figure 4). The timing of the effect is almost always aligned with the song you’ve chosen. For example, you might flash snowflakes off and on in synchronization with the drum beat of a song.
Figure 4: Programming lights.
Music
If this step sounds overwhelming to you, you’re not alone. In fact, I don’t sequence my own songs. I purchase them from different vendors, who create sequences for generic setups with a wide variety of props. I then import them and map them to the different elements that I actually use in my show. In terms of time, professionals can spend many hours to animate one minute of a song. I generally spend about two hours mapping an existing sequence to my show’s layout. My favorite sequence vendors include BF Light Shows, xTreme Sequences, and Magical Light Shows.
Falcon Player
Once you have a sequence built, you use another piece of software to send that sequence to your show controllers. Some controllers have this software built in, but most people I know use another open source application, Falcon Player (FPP), to perform this task. Not only can FPP be run on a Raspberry Pi, but it also is shipped as a Docker image! FPP includes the ability to play back your sequence as well as to build playlists and set up a show schedule for automated playback.
Put it all together and flip the switch
When everything is put together, you should have a system similar to Figure 5:
Although building your own light show may seem like a daunting task, fear not; you are not alone. I have yet to mention the most important part of this whole process: the community. The xLights community is one of the most helpful I’ve ever been part of. You can get questions answered via the official Facebook group as well through as other groups dedicated to specific sequence and controller vendors. Additionally, a Zoom support meeting runs 24×7 and is staffed by hobbyists from across the globe. So, what are you waiting for? Go ahead and start planning your first holiday light show!
We are thrilled to announce that Granite models, IBM’s family of open source and proprietary models built for business, as well as Red Hat InstructLab model alignment tools, are now available on Docker Hub.
Now, developer teams can easily access, deploy, and scale applications using IBM’s AI models specifically designed for developers.
This news will be officially announced during the AI track of the keynote at IBM TechXchange on October 22. Attendees will get an exclusive look at how IBM’s Granite models on Docker Hub accelerate AI-driven application development across multiple programming languages.
Why Granite on Docker Hub?
With a principled approach to data transparency, model alignment, and security, IBM’s open source Granite models represent a significant leap forward in natural language processing. The models are available under an Apache 2.0 license, empowering developer teams to bring generative AI into mission-critical applications and workflows.
Granite models deliver superior performance in coding and targeted language tasks at lower latencies, all while requiring a fraction of the compute resources and reducing the cost of inference. This efficiency allows developers to experiment, build, and scale generative AI applications both on-premises and in the cloud, all within departmental budgetary limits.
Here’s what this means for you:
Simplified deployment: Pull the Granite image from Docker Hub and get up and running in minutes.
Scalability: Docker offers a lightweight and efficient method for scaling artificial intelligence and machine learning (AI/ML) applications. It allows you to run multiple containers on a single machine or distribute them across different machines in a cluster, enabling horizontal scalability.
Flexibility: Customize and extend the model to suit your specific needs without worrying about underlying infrastructure.
Portability: By creating Docker images once and deploying them anywhere, you eliminate compatibility problems and reduce the need for configurations.
Community support: Leverage the vast Docker and IBM communities for support, extensions, and collaborations.
In addition to the IBM Granite models, Red Hat also made the InstructLab model alignment tools available on Docker Hub. Developers using InstructLab can adapt pre-trained LLMs using far less real-world data and computing resources than alternative methodologies. InstructLab is model-agnostic and can be used to fine-tune any LLM of your choice by providing additional skills and knowledge.
With IBM Granite AI models and InstructLab available on Docker Hub, Docker and IBM enable easy integration into existing environments and workflows.
Getting started with Granite
You can find the following images available on Docker Hub:
InstructLab: Ideal for desktop or Mac users looking to explore InstructLab, this image provides a simple introduction to the platform without requiring specialized hardware. It’s perfect for prototyping and testing before scaling up.
Granite-7b-lab: This image is optimized for model serving and inference on desktop or Mac environments, using the Granite-7B model. It allows for efficient and scalable inference tasks without needing a GPU, perfect for smaller-scale deployments or local testing.
How to pull and run IBM Granite images from Docker Hub
IBM Granite provides a toolset for building and managing cloud-native applications. Follow these steps to pull and run an IBM Granite image using Docker and the CLI. You can follow similar steps for the Red Hat InstructLab images.
Authenticate to Docker Hub
Enter your Docker username and password when prompted.
Pull the IBM Granite Image
Pull the IBM Granite image from Docker Hub.
redhat/granite-7b-lab-gguf: For Mac/desktop users with no GPU support
Run the Image in a Container
Start a container with the IBM Granite image. The container can be started in two modes: CLI (default) and server.
To start the container in CLI mode, run the following: docker run --ipc=host -it redhat/granite-7b-lab-gguf
This command opens an interactive bash session within the container, allowing you to use the tools.
To run the container in server mode, run the following command:
docker run --ipc=host -it redhat/granite-7b-lab-gguf -s
Granite on Docker Hub will be officially announced at the IBM TechXchange Conference, which will be held October 21-24 in Las Vegas. Our head of technical alliances, Eli Aleyner, will show a live demonstration at the AI track of the keynote during IBM TechXchange. Oleg Šelajev, Docker’s staff developer evangelist, will show how app developers can test their GenAI apps with local models. Additionally, you’ll learn how Docker’s collaboration with Red Hat is improving developer productivity.
The availability of Granite on Docker Hub marks a significant milestone in making advanced AI models accessible to all. We’re excited to see how developer teams will harness the power of Granite to innovate and solve complex challenges.
Stay anchored for more updates, and as always, happy coding!
Today, we’re excited to announce the release of Docker Build checks with Docker Desktop 4.33. Docker Build checks help your team learn and follow best practices for building container images. When you run a Docker Build, you will get a list of warnings for any check violations detected in your build. Taking a proactive approach and resolving Build warnings and issues early will save you time and headaches downstream.
Why did we create Docker Build checks?
During conversations with developers, we found that many struggle to learn and follow the best practices for building container images. According to our 2024 State of Application Development Report, 35% of Docker users reported creating and editing Dockerfiles as one of the top three tasks performed. However, 55% of respondents reported that creating Dockerfiles is the most selected task they refer to support.
Developers often don’t have the luxury of reading through the Docker Build docs, making the necessary changes to get things working, and then moving on. A Docker Build might “work” when you run docker build, but a poorly written Dockerfiles may introduce quality issues, such as they are:
Hard to maintain or update
Contain hidden and unexpected bugs
Have sub-optimal performance
In our conversations with Docker users, we heard that they want to optimize their Dockerfiles to improve build performance, aren’t aware of current best practices, and would like to be guided as they build.
Investigating and fixing build issues wastes time. We created Docker Build checks to empower developers to write well-structured Dockerfiles from the get-go and learn from existing best practices. With Build checks, your team spends less time on build issues and more on innovation and coding.
Why should you use Docker Build checks?
You want to write better Dockerfiles and save time!
We have collected a set of best practices from the community of build experts and codified them into Docker Build tooling. You can use Docker Build checks to evaluate all stages of your local and CI workflows, including multi-stage builds and Bake, and deep dive in the Docker Desktop Builds view. You can also choose which rules to skip.
You can access Docker Build checks in the CLI and in the Docker Desktop Builds view.
More than just linting: Docker Build checks are powerful and fast
Linting tools typically just evaluate the text files against a set of rules. As a native part of Docker Build, the rules in Docker Build checks are more powerful and accurate than just linting. Docker Build checks evaluate the entire build, including the arguments passed in and the base images used. These checks are quick enough to be run in real-time as you edit your Dockerfile. You can quickly evaluate a build without waiting for a full build execution.
Check your local builds
A good practice is to evaluate a new or updated Dockerfile before committing or sharing your changes. Running docker build will now give you an overview of issues and warnings in your Dockerfile.
Figure 1: A Docker Build with four check warnings displayed.
To get more information about these specific issues, you can specify the debug flag to the Docker CLI with docker --debug build. This information includes the type of warning, where it occurs, and a link to more information on how to resolve it.
Figure 2: Build debug output for the check warnings.
Quickly check your build
Running these checks during a build is great, but it can be time-consuming to wait for the complete build to run each time when you’re making changes or fixing issues. For this reason, we added the --check flag as part of the build command.
# The check flag can be added anywhere as part of your build command
docker build . --check
docker build --check .
docker build --build-arg VERSION=latest --platfrom linux/arm64 . --check
As illustrated in the following figure, appending the flag to your existing build command will do the same full evaluation of the build configuration without executing the entire build. This faster feedback typically completes in less than a second, making for a smoother development process.
Figure 3: Running check of build.
Check your CI builds
By default, running a Docker build with warnings will not cause the build to fail (return a non-zero exit code). However, to catch any regressions in your CI builds, add the following declarations to instruct the checks to generate errors.
# syntax=docker/dockerfile:1
# check=error=true
FROM alpine
CMD echo "Hello, world!"
Checking multi-stage builds in CI
During a build, only the specified stage/target, including its dependent, is executed. We recommend adding a stage check step in your workflow to do a complete evaluation of your Dockerfile. This is similar to how you would run automated tests before executing the full build.
If any warnings are detected, it will return a non-zero exit code, which will cause the workflow to fail, therefore catching any issues.
docker build --check .
Checking builds in Docker Build Cloud
Of course, this also works seamlessly with Docker Build Cloud, both locally and through CI. Use your existing cloud builders to evaluate your builds. Your team now has the combined benefit of Docker Build Cloud performance with the reassurance that the build will align with best practices. In fact, as we expand our checks, you should see even better performance from your Docker Build Cloud builds.
Figure 4: Running checks in Docker Build Cloud.
Configure rules
You have the flexibility to configure rules in Build checks with a skip argument. You can also specify skip=all or skip=none to toggle the rules on and off. Here’s an example of skipping the JSONArgsRecommended and StageNameCasing rules:
# syntax=docker/dockerfile:1
# check=skip=JSONArgsRecommended,StageNameCasing
FROM alpine AS BASE_STAGE
CMD echo "Hello, world!"
Dive deep into Docker Desktop Builds view
In Docker Desktop Builds view, you can see the output of the build warnings. Locating the cause of warnings in Dockerfiles and understanding how to resolve them quickly is now easy.
As with build errors, warnings are shown inline with your Dockerfile when inspecting a build in Docker Desktop:
Figure 5: Build checks warnings in Docker Desktop Builds view.
What’s next?
More checks
We are excited about the new Builds checks to help you apply best practices to your Dockfiles, but this is just the start. In addition to the current set of checks, we plan on adding even more to provide a more comprehensive evaluation of your builds. Further, we look forward to including custom checks and policies for your Docker builds.
IDE integration
The earlier you identify issues in your builds, the easier and less costly it is to resolve them. We plan to integrate Build checks with your favorite IDEs so you can get real-time feedback as you type.
Figure 6: Check violations displaying in VS Code.
GitHub Actions and Docker Desktop
You can already see Build checks warnings in Docker Desktop, but more detailed insights are coming soon to Docker Desktop. As you may have heard, we recently announced Inspecting Docker Builds in GitHub Actions’s beta release, and we plan to build on this new functionality to include support for investigating check warnings.
Get started now
To get started with Docker Build checks, upgrade to Docker Desktop 4.33 today and try them out with your existing Dockerfiles. Head over to our documentation for a more detailed breakdown of Build checks.
Learn more
Authenticate and update to receive your subscription level’s newest Docker Desktop features.
What else is new Docker Desktop 4.33? GA Releases of Docker Debug and Docker Build Checks Plus Enhanced Configuration Integrity Checks.
Docker launched its 2024 State of Application Development Report, providing a deep-focus snapshot of today’s rapidly evolving world of software development. Based on a wide-ranging survey of more than 1,300 respondents, the report shares a broad array of findings about respondents’ work, including what tools they use, their processes and frustrations, opinions about industry trends, participation in developer communities, Docker usage, and more.
What emerges is an illuminating picture of the current state of application development, along with insights into key trends such as the expanding roles of cloud and artificial intelligence/machine learning (AI/ML) in software development, the continued rise of microservices, and attitudes toward the shift-left approach to security.
Reflecting the changing state of the industry, the 2024 report drills down into three main areas:
The state of application development today
AI’s expanding role in application development
Security in application development
The online survey is a key vector through which Docker product managers, engineers, and designers gather insights from users to continuously develop and improve the company’s suite of tools.
“The findings in this report demonstrate how Docker continuously seeks to address market needs so that we can better empower development teams not just to compete, but to thrive and innovate with the right processes and tools for their workflows,” said Nahid Samsami, Vice President of Developer Experience at Docker.
A key insight was the growing popularity of developing software in the cloud. When asked about their main development environment, almost 64% of respondents cited their laptop or desktop. But the real story is that more than 36% cited non-local environments, such as ephemeral environments, personal remote dev environments or clusters, and remote development tools such as GitHub Codespaces, Gitpod, and Coder.
These findings appear to underscore the growing popularity of developing software in the cloud — a trend fanned by benefits such as increased efficiency, shorter build times, reduced time to market, and faster innovation.
Why this apparent increase in reliance on cloud during development? It seems likely that the growing size of applications is a factor, along with the increasing number of dependencies and overall growth in complexity — all of which would render an all-local environment difficult, if not impossible, to maintain in parity with production.
AI/ML goes mainstream in app development
Another key finding was the growing penetration of AI/ML into the software development field. Most respondents (64%) reported already using AI for work — for tasks such as code writing, documentation, and research.
This trend is notably driven by junior/mid-level developers and DevOps/platform engineers, who expressed a higher dependency on AI compared with senior developers surveyed. In terms of AI tools, respondents most often used ChatGPT (46%), GitHub Copilot (30%), and Gemini (formerly Bard) (19%).
This year’s survey showed a growing interest in ML engineering and data science within the Docker community. When asked if they were working on ML in any capacity, almost half of respondents (46%) replied in the affirmative. Within that group, 54% said that they trained and deployed ML models in one or more projects, 43% worked on ML infrastructure, and 39% leveraged pre-trained ML models.
Where developers get stuck
A primary goal of the survey was to gain insights into how Docker can improve the app development experience. When we asked where their team gets stuck in the development process, respondents cited multiple stages, including planning (31%), estimation (24%), and designing (22%). Planning was also one of the most-selected areas in which respondents desired better tools (28% of respondents).
These findings demonstrate that respondents hit sticking points in project-level tasks before development. However, there are areas identified for improvement within the development process itself, as 20% of respondents reported getting stuck during debugging/troubleshooting or testing phases. Testing was also one of the top areas in which respondents wanted better tools (28%).
Microservices, security, and open source
Other notable trends include the continued rise of microservices, frustration with the shift-left approach to security, and interest in open source. Underscoring the growing popularity of microservices, nearly three times more respondents (29%) said they were transitioning from monolithic to microservices than were moving in the other direction, from microservices to monolithic (11%).
The shift-left approach to security appears to be a source of frustration for developers and an area where more effective tools could make a difference. Security-related tasks topped the list of those deemed difficult/very difficult, with 34% of respondents selecting one of these options. Regarding the need for better tools in the development process, 25% of respondents selected security/vulnerability remediation tools (behind testing, planning, and monitoring/logging/maintenance).
Open source software is important to developer ecosystems and communities, with 59% of respondents saying they contributed to open source in the past year, compared with 41% saying they did not. Of the 41% who did not contribute, a large majority (72%) expressed interest in contributing to open source, while less than 25% did not.
Get the full report
The 2024 Docker State of Application Development Report is based on an online, 20-minute survey conducted by Docker’s User Research Team in the fall of 2023. Survey respondents ranged from home hobbyists to professionals at companies with more than 5,000 employees. The findings are based on 885 completed responses from the roughly 1,300 respondents surveyed.
The survey was developed to inform Docker’s product strategy. Given the fascinating information we discovered, we wanted to share the findings with the community. This was the second annual Docker State of Application Development survey; the third will take place in the fall of 2024.
Docker has become a leader in providing software development solutions, offering tooling that simplifies the process of developing, testing, deploying, and running applications using containers. As such, understanding Docker’s different products, like Docker Desktop, and components, like Docker Engine, and understanding how they work together is essential for developers looking to maximize their productivity and ensure compliance with Docker’s licensing terms.
This post will clarify the distinctions and similarities between Docker Desktop and Docker Engine, and provide guidance on verifying which one you are currently using so you can make the most out of your experience.
Read on to explore how to distinguish between Docker Desktop and Docker Engine installations, identify when additional Docker tools are in use, understand Docker contexts, and review your Docker usage to ensure it complies with the Docker Licensing Agreement.
Background
The word “Docker” has become synonymous with containerization to the degree that containerizing an application is increasingly referred to as “dockerizing” an application. Although Docker didn’t create containerization, the company was the first to bring this technology to the developer community in an easily understood set of tooling and artifacts.
In 2015, Docker took the next step and created the Open Container Initiative (OCI) to define and specify how to build a container image, how to run a container image, and how to share container images. By donating the OCI to the Linux Foundation, Docker provided a level playing field for any application of container technology.
An open source effort headed up by Docker is the Moby Project. Docker created this open framework to assemble specialized container systems without reinventing the wheel. It provides a building block set of dozens of standard components and a framework for assembling them into custom platforms.
Moby comprises several components, including a container engine, container runtime, networking, storage, and an orchestration system. Both the standalone free Docker Engine (also known Docker Community Edition or Docker CE) and the commercial Docker Desktop originated from the Moby Project. However, Docker Desktop has evolved beyond the Moby Project, with a full product team investing in the features and technology to support individual developers, small teams, and the requirements of large development teams.
Docker Engine vs. Docker Desktop
Docker Desktop is a commercial product sold and supported by Docker, Inc. It includes the Docker Engine and other open source components; proprietary components; and features like an intuitive GUI, synchronized file shares, access to cloud resources, debugging features, native host integration, governance, and security features that support ECI, air-gapped containers, and administrative settings management. To provide a consistent user experience across different operating systems, Docker Desktop uses the host systems’ native virtualization to run and manage a VM for the Docker Engine. This offers developers a turnkey solution for running a containerization toolset on any device or operating system. Users can leverage the Docker Engine at its core on any platform by downloading Docker Desktop.
Docker Engine is a component free to download individually, not as part of Docker Desktop, and runs as a standalone for free. It can run on any supported Linux distribution and includes the Docker CLI to run commands. Docker Engine will not run natively on Windows or macOS and does not come with a GUI or any of the advanced features provided by Docker Desktop.
How can I tell If I’m running Docker Desktop or just the Docker Engine?
You can determine if you’re using Docker Desktop or Docker Engine in a number of different ways. The following section provides guidance for checking from the filesystem and from within the Docker CLI tooling, which is a component in Docker Desktop as well.
1. GUI or icon
If you are using Docker Desktop, you will have either a windowed GUI or a menubar/taskbar icon of a whale (Figure 1).
Figure 1: The macOS menu.
2. Check for an installation
The easiest way to check for Docker Desktop is to look for the installation; this can be automated by scripting or the use of an MDM solution.
Note that the following instructions assume that Docker Desktop is installed in the default location, which may result in false negatives if other installation paths are used.
Docker Desktop on macOS
On macOS, the Docker Desktop application is installed under the /Applications directory and is named Docker (Figure 2).
Note that the launch icon and location for Docker will depend on the Linux distribution being used.
3. Check a running installation
You can also check a running installation to determine which version of Docker is being used. To do this, you need to use the docker version command; the Server line will indicate the version being used.
Docker Desktop on macOS Arm64
The Server: Docker Desktop 4.24.0 (122432) line indicates using Docker Desktop.
Note that multiple contexts can be installed on a system; this is most often seen in Linux where Docker Desktop and Docker Engine are installed on the same host. To switch between the two, the docker context use command is used. When you are in a context, you communicate with the daemon for that context; thus, in a dual installation situation, you would be switching between the Docker Desktop install and the host install.
To view contexts, you use docker context ls, then switch via docker context use CONTEXTNAME. The following example shows a Linux system with both installed.
$ docker context ls
NAME TYPE DESCRIPTION DOCKER ENDPOINT KUBERNETES ENDPOINT ORCHESTRATOR
default * moby Current DOCKER_HOST based configuration unix:///var/run/docker.sock
desktop-linux moby Docker Desktop unix:///home/jschmidt/.docker/desktop/docker.sock
$ docker version
Client: Docker Engine - Community
Cloud integration: v1.0.35+desktop.5
Version: 24.0.6
API version: 1.43
Go version: go1.20.7
Git commit: ed223bc
Built: Mon Sep 4 12:31:44 2023
OS/Arch: linux/amd64
Context: default
Server: Docker Engine - Community
Engine:
Version: 24.0.6
API version: 1.43 (minimum version 1.12)
Go version: go1.20.7
Git commit: 1a79695
Built: Mon Sep 4 12:31:44 2023
OS/Arch: linux/amd64
Experimental: true
containerd:
Version: 1.6.24
GitCommit: 61f9fd88f79f081d64d6fa3bb1a0dc71ec870523
runc:
Version: 1.1.9
GitCommit: v1.1.9-0-gccaecfc
docker-init:
Version: 0.19.0
GitCommit: de40ad0
$ docker context use desktop-linux
desktop-linux
Current context is now "desktop-linux"
$ docker version
Client: Docker Engine - Community
Cloud integration: v1.0.35+desktop.5
Version: 24.0.6
API version: 1.43
Go version: go1.20.7
Git commit: ed223bc
Built: Mon Sep 4 12:31:44 2023
OS/Arch: linux/amd64
Context: desktop-linux
Server: Docker Desktop 4.24.0 (122432)
Engine:
Version: 24.0.6
API version: 1.43 (minimum version 1.12)
Go version: go1.20.7
Git commit: 1a79695
Built: Mon Sep 4 12:32:16 2023
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.6.22
GitCommit: 8165feabfdfe38c65b599c4993d227328c231fca
runc:
Version: 1.1.8
GitCommit: v1.1.8-0-g82f18fe
docker-init:
Version: 0.19.0
GitCommit: de40ad0
Other OCI tooling
Because both Docker Engine and Docker Desktop are OCI compliant, a number of solutions are presented and installed as “direct replacements” for Docker. This process usually involves creating helper aliases, scripts, or batch programs to emulate docker commands.
You can check for aliases by running the command alias docker to see if there is an alias in place. This holds true for Linux and macOS, or a Linux distribution inside WSL2 on Windows.
$ alias docker # Docker aliased to podman
docker='podman'
$ alias docker # No alias present
You can also list the docker binary from the CLI to ensure that it is the official Docker binary:
$ ls -l `which docker` # Docker supplied by Homebrew on the Mac
lrwxr-xr-x 1 jschmidt admin 34 Apr 2 12:03 /opt/homebrew/bin/docker -> ../Cellar/docker/26.0.0/bin/docker
$ ls -l `which docker` # Official Docker binary on the Mac
lrwxr-xr-x 1 root wheel 54 Jan 10 16:06 /usr/local/bin/docker -> /Applications/Docker.app/Contents/Resources/bin/docker
Conclusion
To wrap up our exploration, note that there are also several offerings generically referred to as “Docker” available to use as part of your containerization journey. This post focused on Docker Engine and Docker Desktop.
At this point, you should be comfortable distinguishing between a Docker Desktop installation and a Docker Engine installation and be able to identify when other OCI tooling is being used under the docker command name. You should also have a high-level understanding of Docker contexts as they relate to this topic. Finally, you should be able to review your usage against the Docker Licensing Agreement to ensure compliance or simply log in with your company credentials to get access to your procured entitlements..
Docker Official Images are a curated set of Docker repositories hosted on Docker Hub that provide a wide range of pre-configured images for popular language runtimes and frameworks, cloud-first utilities, data stores, and Linux distributions. These images are maintained and vetted, ensuring they meet best practices for security, usability, and versioning, making it easier for developers to deploy and run applications consistently across different environments.
Docker Official Images are an important component of Docker’s commitment to the security of both the software supply chain and open source software. Docker Official Images provide thousands of images you can use directly or as a base image when building your own images. For example, there are Docker Official Images for Alpine Linux, NGINX, Ubuntu, PostgreSQL, Python, and Node.js. Visit Docker Hub to search through the currently available Docker Official Images.
In this blog post, we address three common misconceptions about Docker Official Images and outline seven ways they help secure the software supply chain.
3 common misconceptions about Docker Official Images
Even though Docker Official Images have been around for more than a decade and have been used billions of times, they are somewhat misunderstood. Who “owns” Docker Official Images? What is with all those tags? How should you use Docker Official Images? Let’s address some of the more common misconceptions.
Misconception 1: Docker Official Images are controlled by Docker
Docker Official Images are maintained through a partnership between upstream maintainers, community volunteers, and Docker engineers. External developers maintain the majority of Docker Official Images Dockerfiles, with Docker engineers providing insight and review to ensure best practices and uniformity across the Docker Official Images catalog. Additionally, Docker provides and maintains the Docker Official Images build infrastructure and logic, ensuring consistent and secure build environments that allow Docker Official Images to support more than 10 architecture/operating system combinations.
Misconception 2: Docker Official Images are designed for a single use case
Most Docker Official Images repositories offer several image variants and maintain multiple supported versions. In other words, the latest tag of a Docker Official Image might not be the right choice for your use case.
Docker Official Images tags
The documentation for each Docker Official Images repository contains a “Supported tags and respective Dockerfile links” section that lists all the current tags with links to the Dockerfiles that created the image with those tags (Figure 1). This section can be a little intimidating for first-time users, but keeping in mind a few conventions will allow even novices to understand what image variants are available and, more importantly, which variant best fits their use case.
Figure 1: Documentation showing the current tags with links to the Dockerfiles that created the image with those tags.
Tags listed on the same line all refer to the same underlying image. (Multiple tags can point to the same image.) For example, Figure 1 shows the ubuntu Docker Official Images repository, where the 20.04, focal-20240216, and focal tags all refer to the same image.
Often the latest tag for a Docker Official Images repository is optimized for ease of use and includes a wide variety of software helpful, but not strictly necessary, when using the main software packaged in the Docker Official Image. For example, latest images often include tools like Git and build tools. Because of their ease of use and wide applicability, latest images are often used in getting-started guides.
Some operating system and language runtime repositories offer “slim” variants that have fewer packages installed and are therefore smaller. For example, the python:3.12.2-bookworm image contains not only the Python runtime, but also any tool you might need to build and package your Python application — more than 570 packages! Compare this to the python:3.12.2-slim-bookworm image, which has about 150 packages.
Many Docker Official Images repositories offer “alpine” variants built on top of the Alpine Linux distribution rather than Debian or Ubuntu. Alpine Linux is focused on providing a small, simple, and secure base for container images, and Docker Official Images alpine variants typically aim to install only necessary packages. As a result, Docker Official Images alpine variants are typically even smaller than “slim” variants. For example, the linux/amd64 node:latest image is 382 MB, the node:slim image is 70 MB, and the node:alpine image is 47 MB.
If you see tags with words that look like Toy Story characters (for example, bookworm, bullseye, and trixie) or adjectives (such as jammy, focal, and bionic), those indicate the codename of the Linux distribution they use as a base image. Debian-release codenames are based on Toy Story characters, and Ubuntu releases use alliterative adjective-animal appellations. Linux distribution indicators are helpful because many Docker Official Images provide variants built upon multiple underlying distribution versions (for example, postgres:bookworm and postgres:bullseye).
Tags may contain other hints to the purpose of their image variant. Often these are explained later in the Docker Official Images repository documentation. Check the “How to use this image” and/or “Image Variants” sections.
Misconception 3: Docker Official Images do not follow software development best practices
Some critics argue that Docker Official Images go against the grain of best practices, such as not running container processes as root. While it’s true that we encourage users to embrace a few opinionated standards, we also recognize that different use cases require different approaches. For example, some use cases may require elevated privileges for their workloads, and we provide options for them to do so securely.
7 ways Docker Official Images help secure the software supply chain
We recognize that security is a continuous process, and we’re committed to providing the best possible experience for our users. Since the company’s inception in 2013, Docker has been a leader in the software supply chain, and our commitment to security — including open source security — has helped to protect developers from emerging threats all along the way.
With the availability of open source software, efficiently building powerful applications and services is easier than ever. The transparency of open source allows unprecedented insight into the security posture of the software you create. But to take advantage of the power and transparency of open source software, fully embracing software supply chain security is imperative. A few ways Docker Official Images help developers build a more secure software supply chain include:
Open build process
Because visibility is an important aspect of the software supply chain, Docker Official Images are created from a transparent and open build process. The Dockerfile inputs and build scripts are all open source, all Docker Official Images updates go through a public pull request process, and the logs from all Docker Official Images builds are available to inspect (Jenkins / GitHub Actions).
Principle of least privilege
The Docker Official Images build system adheres strictly to the principle of least privilege (POLP), for example, by restricting writes for each architecture to architecture-specific build agents.
Updated build system
Ensuring the security of Docker Official Images builds and images is paramount. The Docker Official Images build system is kept up to date through automated builds, regular security audits, collaboration with upstream projects, ongoing testing, and security patches.
Vulnerability reports and continuous monitoring
Courtesy of Docker Scout, vulnerability insights are available for all Docker Official Images and are continuously updated as new vulnerabilities are discovered. We are committed to continuously monitoring our images for security issues and addressing them promptly. For example, we were among the first to provide reasoned guidance and remediation for the recent xz supply chain attack. We also use insights and remediation guidance from Docker Scout, which surfaces actionable insights in near-real-time by updating CVE results from 20+ CVE databases every 20-60 minutes.
Software Bill of Materials (SBOM) and provenance attestations
We are committed to providing a complete and accurate SBOM and detailed build provenance as signed attestations for all Docker Official Images. This allows our users to have confidence in the origin of Docker Official Images and easily identify and mitigate any potential vulnerabilities.
Signature validation
We are working on integrating signature validation into our image pull and build processes. This will ensure that all Docker Official Images are verified before use, providing an additional layer of security for our users.
Increased update frequency
Docker Official Images provide the best of both worlds: the latest version of the software you want, built upon stable versions of Linux distributions. This allows you to use the latest features and fixes of the software you are running without having to wait for a new package from your Linux distribution or being forced to use an unstable version of your Linux distribution. Further, we are working to increase the throughput of the Docker Official Images build infrastructure to allow us to support more frequent updates for larger swaths of Docker Official Images. As part of this effort, we are piloting builds on GitHub Actions and Docker Build Cloud.
Conclusion
Docker’s leadership in security and protecting open source software has been established through Docker Official Images and other trusted content we provide our customers. We take a comprehensive approach to security, focusing on best practices, tooling, and community engagement, and we work closely with upstream projects and SIGs to address security issues promptly and proactively.
Docker Official Images provide a flexible and secure way for developers to build, ship, test, and run their applications. Docker Official Images are maintained through a partnership between the Docker Official Images community, upstream maintainers/volunteers, and Docker engineers, ensuring best practices and uniformity across the Docker Official Images catalog. Each Docker Official Image offers numerous image variants that cater to different use cases, with tags indicating the purpose of each variant.
Developers can build using Docker tools and products with confidence, knowing that their applications are built on a secure, transparent foundation.
Looking to dive in? Get started building with Docker Official Images today.
One of the challenges in modern software development is being able to release software often and with confidence. This can only be achieved when you have a good CI/CD setup in place that can test your software and release it with minimal or even no human intervention. But modern software applications also use a wide range of third-party dependencies and often need to run on multiple operating systems and architectures.
Bazel is an open source build tool developed by Google to build and test multi-language, multi-platform projects. Several big IT companies have adopted monorepos for various reasons, such as:
Code sharing and reusability
Cross-project refactoring
Consistent builds and dependency management
Versioning and release management
With its multi-language support and focus on reproducible builds, Bazel shines in building such monorepos.
A key concept of Bazel is hermeticity, which means that when all inputs are declared, the build system can know when an output needs to be rebuilt. This approach brings determinism where, given the same input source code and product configuration, it will always return the same output by isolating the build from changes to the host system.
Testcontainers is an open source framework for provisioning throwaway, on-demand containers for development and testing use cases. Testcontainers make it easy to work with databases, message brokers, web browsers, or just about anything that can run in a Docker container.
Using Bazel and Testcontainers together offers the following features:
Bazel can build projects using different programming languages like C, C++, Java, Go, Python, Node.js, etc.
Bazel can dynamically provision the isolated build/test environment with desired language versions.
Testcontainers can provision the required dependencies as Docker containers so that your test suite is self-contained. You don’t have to manually pre-provision the necessary services, such as databases, message brokers, and so on.
All the test dependencies can be expressed through code using Testcontainers APIs, and you avoid the risk of breaking hermeticity by sharing such resources between tests.
Let’s see how we can use Bazel and Testcontainers to build and test a monorepo with modules using different languages. We are going to explore a monorepo with a customers module, which uses Java, and a products module, which uses Go. Both modules interact with relational databases (PostgreSQL) and use Testcontainers for testing.
Getting started with Bazel
To begin, let’s get familiar with Bazel’s basic concepts. The best way to install Bazel is by using Bazelisk. Follow the official installation instructions to install Bazelisk. Once it’s installed, you should be able to run the Bazelisk version and Bazel version commands:
Before you can build a project using Bazel, you need to set up its workspace.
A workspace is a directory that holds your project’s source files and contains the following files:
The WORKSPACE.bazel file, which identifies the directory and its contents as a Bazel workspace and lives at the root of the project’s directory structure.
A MODULE.bazel file, which declares dependencies on Bazel plugins (called “rulesets”).
One or more BUILD (or BUILD.bazel) files, which describe the sources and dependencies for different parts of the project. A directory within the workspace that contains a BUILD file is a package.
In the simplest case, a MODULE.bazel file can be an empty file, and a BUILD file can contain one or more generic targets as follows:
Here, we have two targets: foo and bar. Now we can build those targets using Bazel as follows:
$ bazel build //:foo <- runs only foo target, // indicates root workspace
$ bazel build //:bar <- runs only bar target
$ bazel build //... <- runs all targets
Configuring the Bazel build in a monorepo
We are going to explore using Bazel in the testcontainers-bazel-demo repository. This repository is a monorepo with a customers module using Java and a products module using Go. Its structure looks like the following:
Bazel uses different rules for building different types of projects. Bazel uses rules_java for building Java packages, rules_go for building Go packages, rules_python for building Python packages, etc.
We may also need to load additional rules providing additional features. For building Java packages, we may want to use external Maven dependencies and use JUnit 5 for running tests. In that case, we should load rules_jvm_external to be able to use Maven dependencies.
We are going to use Bzlmod, the new external dependency subsystem, to load the external dependencies. In the MODULE.bazel file, we can load the additional rules_jvm_external and contrib_rules_jvm as follows:
Let’s understand the above configuration in the MODULE.bazel file:
We have loaded the rules_jvm_external rules fromBazel Central Registry and loaded extensions to use third-party Maven dependencies.
We have configured all our Java application dependencies using Maven coordinates in the maven.install artifacts configuration.
We are loading the contrib_rules_jvm rules that supports running JUnit 5 tests as a suite.
Now, we can run the @maven//:pin program to create a JSON lockfile of the transitive dependencies, in a format that rules_jvm_external can use later:
bazel run @maven//:pin
Rename the generated file rules_jvm_external~4.5~maven~maven_install.json to maven_install.json. Now update the MODULE.bazel to reflect that we pinned the dependencies.
Add a lock_file attribute to the maven.install() and update the use_repo call to also expose the unpinned_maven repository used to update the dependencies:
We have loaded default_java_toolchain and then configured the Java version to 17.
We have configured a java_library target with the name customers-lib that will build the production jar file.
We have defined a java_test_suite target with the name customers-lib-tests to define our test suite, which will execute all the tests. We also configured the dependencies on the other target customers-lib and external dependencies.
We also defined another target with the name customers-test-resources to add non-Java sources (e.g., logging config files) to our test suite target as a dependency.
In the customers package, we have a CustomerService class that stores and retrieves customer details in a PostgreSQL database. And we have CustomerServiceTest that tests CustomerService methods using Testcontainers. Take a look at the GitHub repository for the complete code.
Note: You can use Gazelle, which is a Bazel build file generator, to generate the BUILD.bazel files instead of manually writing them.
Running Testcontainers tests
For running Testcontainers tests, we need a Testcontainers-supported container runtime. Let’s assume you have a local Docker installed using Docker Desktop.
Now, with our Bazel build configuration, we are ready to build and test the customers package:
# to run all build targets of customers package
$ bazel build //customers/...
# to run a specific build target of customers package
$ bazel build //customers:customers-lib
# to run all test targets of customers package
$ bazel test //customers/...
# to run a specific test target of customers package
$ bazel test //customers:customers-lib-tests
When you run the build for the first time, it will take time to download the required dependencies and then execute the targets. But, if you try to build or test again without any code or configuration changes, Bazel will not re-run the build/test again and will show the cached result. Bazel has a powerful caching mechanism that will detect code changes and run only the targets that are necessary to run.
While using Testcontainers, you define the required dependencies as part of code using Docker image names along with tags, such as Postgres:16. So, unless you change the code (e.g., Docker image name or tag), Bazel will cache the test results.
Similarly, we can use rules_go and Gazelle for configuring Bazel build for Go packages. Take a look at the MODULE.bazel and products/BUILD.bazel files to learn more about configuring Bazel in a Go package.
As mentioned earlier, we need a Testcontainers-supported container runtime for running Testcontainers tests. Installing Docker on complex CI platforms might be challenging, and you might need to use a complex Docker-in-Docker setup. Additionally, some Docker images might not be compatible with the operating system architecture (e.g., Apple M1).
Testcontainers Cloud solves these problems by eliminating the need to have Docker on the localhost or CI runners and run the containers on cloud VMs transparently.
Here is an example of running the Testcontainers tests using Bazel on Testcontainers Cloud using GitHub Actions:
GitHub Actions runners already come with Bazelisk installed, so we can use Bazel out of the box. We have configured the TC_CLOUD_TOKEN environment variable through Secrets and started the Testcontainers Cloud agent. If you check the build logs, you can see that the tests are executed using Testcontainers Cloud.
Summary
We have shown how to use the Bazel build system to build and test monorepos with multiple modules using different programming languages. Combined with Testcontainers, you can make the builds self-contained and hermetic.
Although Bazel and Testcontainers help us have a self-contained build, we need to take extra measures to make it a hermetic build:
Bazel can be configured to use a specific version of SDK, such as JDK 17, Go 1.20, etc., so that builds always use the same version instead of what is installed on the host machine.
For Testcontainers tests, using Docker tag latestfor container dependencies may result in non-deterministic behavior. Also, some Docker image publishers override the existing images using the same tag. To make the build/test deterministic, always use the Docker image digest so that the builds and tests always use the exact same version of images that gives reproducible and hermetic builds.
Using Testcontainers Cloud for running Testcontainers tests reduces the complexity of Docker setup and gives a deterministic container runtime environment.
With Nix Shell, you can instantly spin up the exact environment you need, tailored to specific projects, and discard it with ease once you’re done. No containers!
We’ll walk you through the installation of Nix and various ways to craft custom environments with Nix Shell.
▬▬▬▬▬▬ Sponsorships ▬▬▬▬▬▬ If you are interested in sponsoring this channel, please use https://calendly.com/vfarcic/meet to book a timeslot that suits and we’ll go over the details. Or feel free to contact me over Twitter or LinkedIn (see below)
This post was contributed by Thierry Moreau, co-founder and head of DevRel at OctoAI.
Generative AI models have shown immense potential over the past year with breakthrough models like GPT3.5, DALL-E, and more. In particular, open source foundational models have gained traction among developers and enterprise users who appreciate how customizable, cost-effective, and transparent these models are compared to closed-source alternatives.
In this article, we’ll explore how you can compose an open source foundational model into a streamlined image transformation pipeline that lets you manipulate images with nothing but text to achieve surprisingly good results.
With this approach, you can create fun versions of corporate logos, bring your kids’ drawings to life, enrich your product photography, or even remodel your living room (Figure 1).
Figure 1: Examples of image transformation including, from left to right: Generating creative corporate logo, bringing children’s drawings to life, enriching commercial photography, remodeling your living room
Pretty cool, right? Behind the scenes, a lot needs to happen, and we’ll walk step by step through how to reproduce these results yourself. We call the multimodal GenAI pipeline OctoShop as a nod to the popular image editing software.
Feeling inspired to string together some foundational GenAI models? Let’s dive into the technology that makes this possible.
Architecture overview
Let’s look more closely at the open source foundational GenAI models that compose the multimodal pipeline we’re about to build.
Going forward, we’ll use the term “model cocktail” instead of “multimodal GenAI model pipeline,” as it flows a bit better (and sounds tastier, too). A model cocktail is a mix of GenAI models that can process and generate data across multiple modalities: text and images are examples of data modalities across which GenAI models consume and produce data, but the concept can also extend to audio and video (Figure 2).
To build on the analogy of crafting a cocktail (or mocktail, if you prefer), you’ll need to mix ingredients, which, when assembled, are greater than the sum of their individual parts.
Figure 2: The multimodal GenAI workflow — by taking an image and text, this pipeline transforms the input image according to the text prompt.
CLIP Interrogator takes in an image and generates a textual description (e.g., “a whale with a container on its back”).
An LLM model, Mistral-7B, will generate a richer textual description based on a user prompt (e.g., “set the image into space”). The LLM will consequently transform the description into a richer one that meets the user prompt (e.g., “in the vast expanse of space, a majestic whale carries a container on its back”).
Finally, an SDXL model will be used to generate a final AI-generated image based on the textual description transformed by the LLM model. We also take advantage of SDXL styles and a ControlNet to better control the output of the image in terms of style and framing/perspective.
Prerequisites
Let’s go over the prerequisites for crafting our cocktail.
Here’s what you’ll need:
Sign up for an OctoAI account to use OctoAI’s image generation (SDXL), text generation (Mistral-7B), and compute solutions (CLIP Interrogator) — OctoAI serves as the bar from which to get all of the ingredients you’ll need to craft your model cocktail. If you’re already using a different compute service, feel free to bring that instead.
Run a Jupyter notebook to craft the right mix of GenAI models. This is your place for experimenting and mixing, so this will be your cocktail shaker. To make it easy to run and distribute the notebook, we’ll use Google Colab.
Finally, we’ll deploy our model cocktail as a Streamlit app. Think of building your app and embellishing the frontend as the presentation of your cocktail (e.g., glass, ice, and choice of garnish) to enhance your senses.
Getting started with OctoAI
Head to octoai.cloud and create an account if you haven’t done so already. You’ll receive $10 in credits upon signing up for the first time, which should be sufficient for you to experiment with your own workflow here.
We’ve built a Jupyter notebook in Colab to help you learn how to use the different models that will constitute your model cocktail. Here are the steps to follow:
There’s no need to change the runtime type or rely on a GPU or TPU accelerator — all we need is a CPU here, given that all of the AI heavy-lifting is done on OctoAI endpoints.
2. OctoAI SDK setup
Let’s get started by installing the OctoAI SDK. You’ll use the SDK to invoke the different open source foundational models we’re using, like SDXL and Mistral-7B. You can install through pip:
# Install the OctoAI SDK
!pip install octoai-sdk
In some cases, you may get a message about pip packages being previously imported in the runtime, causing an error. If that’s the case, selecting the Restart Session button at the bottom should take care of the package versioning issues. After this, you should be able to re-run the cell that pip-installs the OctoAI SDK without any issues.
3. Generate images with SDXL
You’ll first learn to generate an image with SDXL using the Image Generation solution API. To learn more about what each parameter does in the code below, check out OctoAI’s ImageGenerator client.
In particular, the ImageGenerator API takes several arguments to generate an image:
Engine: Lets you choose between versions of Stable Diffusion models, such as SDXL, SD1.5, and SSD.
Prompt: Describes the image you want to generate.
Negative prompt: Describes the traits you want to avoid in the final image.
Width, height: The resolution of the output image.
Num images: The number of images to generate at once.
Sampler: Determines the sampling method used to denoise your image. If you’re not familiar with this process, this article provides a comprehensive overview.
Number of steps: Number of denoising steps — the more steps, the higher the quality, but generally going past 30 will lead to diminishing returns.
Cfg scale: How closely to adhere to the image description — generally stays around 7-12.
Use refiner: Whether to apply the SDXL refiner model, which improves the output quality of the image.
Seed: A parameter that lets you control the reproducibility of image generation (set to a positive value to always get the same image given stable input parameters).
Note that tweaking the image generation parameters — like number of steps, number of images, sampler used, etc. — affects the amount of GPU compute needed to generate an image. Increasing GPU cycles will affect the pricing of generating the image.
Here’s an example using simple parameters:
# To use OctoAI, we'll need to set up OctoAI to use it
from octoai.clients.image_gen import Engine, ImageGenerator
# Now let's use the OctoAI Image Generation API to generate
# an image of a whale with a container on its back to recreate
# the moby logo
image_gen = ImageGenerator(token=OCTOAI_API_TOKEN)
image_gen_response = image_gen.generate(
engine=Engine.SDXL,
prompt="a whale with a container on its back",
negative_prompt="blurry photo, distortion, low-res, poor quality",
width=1024,
height=1024,
num_images=1,
sampler="DPM_PLUS_PLUS_2M_KARRAS",
steps=20,
cfg_scale=7.5,
use_refiner=True,
seed=1
)
images = image_gen_response.images
# Display generated image from OctoAI
for i, image in enumerate(images):
pil_image = image.to_pil()
display(pil_image)
Feel free to experiment with the parameters to see what happens to the resulting image. In this case, I’ve put in a simple prompt meant to describe the Docker logo: “a whale with a container on its back.” I also added standard negative prompts to help generate the style of image I’m looking for. Figure 3 shows the output:
Figure 3: An SDXL-generated image of a whale with a container on its back.
4. Control your image output with ControlNet
One thing you may want to do with SDXL is control the composition of your AI-generated image. For example, you can specify a specific human pose or control the composition and perspective of a given photograph, etc.
For our experiment using Moby (the Docker mascot), we’d like to get an AI-generated image that can be easily superimposed onto the original logo — same shape of whale and container, orientation of the subject, size, and so forth.
This is where ControlNet can come in handy: they let you constrain the generation of images by feeding a control image as input. In our example we’ll feed the image of the Moby logo as our control input.
By tweaking the following parameters used by the ImageGenerator API, we are constraining the SDXL image generation with a control image of Moby. That control image will be converted into a depth map using a depth estimation model, then fed into the ControlNet, which will constrain SDXL image generation.
# Set the engine to controlnet SDXL
engine="controlnet-sdxl",
# Select depth controlnet which uses a depth map to apply
# constraints to SDXL
controlnet="depth_sdxl",
# Set the conditioning scale anywhere between 0 and 1, try different
# values to see what they do!
controlnet_conditioning_scale=0.3,
# Pass in the base64 encoded string of the moby logo image
controlnet_image=image_to_base64(moby_image),
Now the result looks like it matches the Moby outline a lot more closely (Figure 4). This is the power of ControlNet. You can adjust the strength by varying the controlnet_conditioning_scale parameter. This way, you can make the output image more or less faithfully match the control image of Moby.
Figure 4: Left: The Moby logo is used as a control image to a ControlNet. Right: the SDXL-generated image resembles the control image more closely than in the previous example.
5. Control your image output with SDXL style presets
Figure 5: You can try various styles on the OctoAI Image Generation solution UI — there are more than 100 to choose from, each delivering a unique feel and aesthetic.
Figure 6 shows how setting this one parameter in the ImageGenerator API transforms our AI-generated image of Moby. Go ahead and try out more styles; we’ve generated a gallery for you to get inspiration from.
Figure 6: SDXL-generated image of Moby with the “3D Model” style preset applied.
6. Manipulate images with Mistral-7B LLM
So far we’ve relied on SDXL, which does text-to-image generation. We’ve added ControlNet in the mix to apply a control image as a compositional constraint.
Next, we’re going to layer an LLM into the mix to transform our original image prompt into a creative and rich textual description based on a “transformation prompt.”
Basically, we’re going to use an LLM to make our prompt better automatically. This will allow us to perform image manipulation using text in our OctoShop model cocktail pipeline:
Take a logo of Moby: Set it into an ultra-realistic photo in space.
Take a child’s drawing: Bring it to life in a fantasy world.
Take a photo of a cocktail: Set it on a beach in Italy.
Take a photo of a living room: Transform it into a staged living room in a designer house.
To achieve this text-to-text transformation, we will use the LLM user prompt as follows. This sets the original textual description of Moby into a new setting: the vast expanse of space.
'''
Human: set the image description into space: “a whale with a container on its back”
AI: '''
We’ve configured the LLM system prompt so that LLM responses are concise and at most one sentence long. We could make them longer, but be aware that the prompt consumed by SDXL has a 77-token context limit.
Model: Lets you choose out of selection of foundational open source models like Mixtral, Mistral, Llama2, Code Llama (the selection will grow with more open source models being released).
Messages: Contains a list of messages (system and user) to use as context for the completion.
Max tokens: Enforces a hard limit on output tokens (this could cut a completion response in the middle of a sentence).
Temperature: Lets you control the creativity of your answer: with a higher temperature, less likely tokens can be selected.
The choice of model, input, and output tokens will influence pricing on OctoAI. In this example, we’re using the Mistral-7B LLM, which is a great open source LLM model that really packs a punch given its small parameter size.
Let’s look at the code used to invoke our Mistral-7B LLM:
# Let's go ahead and start with the original prompt that we used in our
# image generation examples.
image_desc = "a whale with a container on its back"
# Let's then prepare our LLM prompt to manipulate our image
llm_prompt = '''
Human: set the image description into space: {}
AI: '''.format(image_desc)
# Now let's use an LLM to transform this craft clay rendition
# of Moby into a fun scify universe
from octoai.client import Client
client = Client(OCTOAI_API_TOKEN)
completion = client.chat.completions.create(
messages=[
{
"role": "system",
"content": "You are a helpful assistant. Keep your responses short and limited to one sentence."
},
{
"role": "user",
"content": llm_prompt
}
],
model="mistral-7b-instruct-fp16",
max_tokens=128,
temperature=0.01
)
# Print the message we get back from the LLM
llm_image_desc = completion.choices[0].message.content
print(llm_image_desc)
Here’s the output:
Our LLM has created a short yet imaginative description of Moby traveling through space. Figure 7 shows the result when we feed this LLM-generated textual description into SDXL.
Figure 7: SDXL-generated image of Moby where we used an LLM to set the scene in space and enrich the text prompt.
This image is great. We can feel the immensity of space. With the power of LLMs and the flexibility of SDXL, we can take image creation and manipulation to new heights. And the great thing is, all we need to manipulate those images is text; the GenAI models do the rest of the work.
7. Automate the workflow with AI-based image labeling
So far in our image transformation pipeline, we’ve had to manually label the input image to our OctoShop model cocktail. Instead of just passing in the image of Moby, we had to provide a textual description of that image.
Thankfully, we can rely on a GenAI model to perform text labeling tasks: CLIP Interrogator. Think of this task as the reverse of what SDXL does: It takes in an image and produces text as the output.
To get started, we’ll need a CLIP Interrogator model running behind an endpoint somewhere. There are two ways to get a CLIP Interrogator model endpoint on OctoAI. If you’re just getting started, we recommend the simple approach, and if you feel inspired to customize your model endpoint, you can use the more advanced approach. For instance, you may be interested in trying out the more recent version of CLIP Interrogator.
You can now invoke the CLIP Interrogator model in a few lines of code. We’ll use the fast interrogator mode here to get a label generated as quickly as possible.
# Let's go ahead and invoke the CLIP interrogator model
# Note that under a cold start scenario, you may need to wait a minute or two
# to get the result of this inference... Be patient!
output = client.infer(
endpoint_url=CLIP_ENDPOINT_URL+'/predict',
inputs={
"image": image_to_base64(moby_image),
"mode": "fast"
}
)
# All labels
clip_labels = output["completion"]["labels"]
print(clip_labels)
# Let's get just the top label
top_label = clip_labels.split(',')[0]
print(top_label)
The top label described our Moby logo as:
That’s pretty on point. Now that we’ve tested all ingredients individually, let’s assemble our model cocktail and test it on interesting use cases.
8. Assembling the model cocktail
Now that we have tested our three models (CLIP interrogator, Mistral-7B, SDXL), we can package them into one convenient function, which takes the following inputs:
An input image that will be used to control the output image and also be automatically labeled by our CLIP interrogator model.
A transformation string that describes the transformation we want to apply to the input image (e.g., “set the image description in space”).
A style string which lets us better control the artistic output of the image independently of the transformation we apply to it (e.g., painterly style vs. cinematic).
The function below is a rehash of all of the code we’ve introduced above, packed into one function.
Now you can try this out on several images, prompts, and styles.
Package your model cocktail into a web app
Now that you’ve mixed your unique GenAI cocktail, it’s time to pour it into a glass and garnish it, figuratively speaking. We built a simple Streamlit frontend that lets you deploy your unique OctoShop GenAI model cocktail and share the results with your friends and colleagues (Figure 8). You can check it on GitHub.
Figure 8: The Streamlit app transforms images into realistic renderings in space — all thanks to the magic of GenAI.
We look forward to seeing what great image-processing apps you come up with. Go ahead and share your creations on OctoAI’s Discord server in the #built_with_octo channel!
If you want to learn how you can put OctoShop behind a Discord Bot or build your own model containers with Docker, we also have instructions on how to do that from an AI/ML workshop organized by OctoAI at DockerCon 2023.
About OctoAI
OctoAI provides infrastructure to run GenAI at scale, efficiently, and robustly. The model endpoints that OctoAI delivers to serve models like Mixtral, Stable Diffusion XL, etc. all rely on Docker to containerize models and make them easier to serve at scale.
If you go to octoai.cloud, you’ll find three complementary solutions that developers can build on to bring their GenAI-powered apps and pipelines into production.
Image Generation solution exposes endpoints and APIs to perform text to image, image to image tasks built around open source foundational models such as Stable Diffusion XL or SSD.
Text Generation solution exposes endpoints and APIs to perform text generation tasks built around open source foundational models, such as Mixtral/Mistral, Llama2, or CodeLlama.
Compute solution lets you deploy and manage any dockerized model container on capable OctoAI cloud endpoints to power your demanding GenAI needs. This compute service complements the image generation and text generation solutions by exposing infinite programmability and customizability for AI tasks that are not currently readily available on either the image generation or text generation solutions.
Disclaimer
OctoShop is built on the foundation of CLIP Interrogator andSDXL, and Mistral-7B and is therefore likely to carry forward the potential dangers inherent in these base models. It’s capable of generating unintended, unsuitable, offensive, and/or incorrect outputs. We therefore strongly recommend exercising caution and conducting comprehensive assessments before deploying this model into any practical applications.
This GenAI model workflow doesn’t work on people as it won’t preserve their likeness; the pipeline works best on scenes, objects, or animals. Solutions are available to address this problem, such as face mapping techniques (also known as face swapping), which we can containerize with Docker and deploy on OctoAI Compute solution, but that’s something to cover in another blog post.
Conclusion
This article covered the fundamentals of building a GenAI model cocktail by relying on a combination of text generation, image generation, and compute solutions powered by the portability and scalability enabled by Docker containerization.
If you’re interested in learning more about building these kinds of GenAI model cocktails, check out the OctoAI demo page or join OctoAI on Discord to see what people have been building.
Acknowledgements
The authors acknowledge Justin Gage for his thorough review, as well as Luis Vega, Sameer Farooqui, and Pedro Toruella for their contributions to the DockerCon AI/ML Workshop 2023, which inspired this article. The authors also thank Cia Bodin and her daughter Ada for the drawing used in this blog post.
Take this DockerCon 2023 crash course on the latest generative AI developments in LLMs and Image Generation models. We provide an overview of techniques for ...
Docker Compose‘s simplicity — just run compose up — has been an integral part of developer workflows for a decade, with the first commit occurring in 2013, back when it was called Plum. Although the feature set has grown dramatically in that time, maintaining that experience has always been integral to the spirit of Compose.
In this post, we’ll walk through how to manage microservice sprawl with Docker Compose by importing subprojects from other Git repos.
Maintaining simplicity
Now, perhaps more than ever, that simplicity is key. The complexity of modern software development is undeniable regardless of whether you’re using microservices or a monolith, deploying to the cloud or on-prem, or writing in JavaScript or C.
Compose has not kept up with this “development sprawl” and is even sometimes an obstacle when working on larger, more complex projects. Maintaining Compose to accurately represent your increasingly complex application can require its own expertise, often resulting in out-of-date configuration in YAML or complex makefile tasks.
As an open source project, Compose serves everyone from home lab enthusiasts to transcontinental corporations, which is no small feat, and our commitment to maintaining Compose’s signature simplicity for all users hasn’t changed.
The increased flexibility afforded by Compose watch and include means your project no longer needs to be one-size-fits-all. Now, it’s possible to split your project across Git repos and import services as needed, customizing their configuration in the process.
Application architecture
Let’s take a look at a hypothetical application architecture. To begin, the application is split across two Git repos:
backend — Backend in Python/Flask
frontend — Single-page app (SPA) frontend in JavaScript/Node.js
While working on the frontend, the developers run without using Docker or Compose, launching npm start on their laptops directly and proxy API requests to a shared staging server (as opposed to running the backend locally). Meanwhile, while working on the backend, developers and CI (for integration tests) share a Compose file and rely on command-line tools like cURL to manually test functionality locally.
We’d like a flexible configuration that enables each group of developers to use their optimal workflow (e.g., leveraging hot reload for the frontend) while also allowing reuse to share project configuration between repos. At first, this seems like an impossible situation to resolve.
Frontend
We can start by adding a compose.yaml file to frontend:
Note: If you’re wondering what the Dockerfile looks like, take a look at this samples page for an up-to-date example of best practices generated by docker init.
This is a great start! Running docker compose up will now build the Node.js frontend and make it accessible at http://localhost:8000/.
The BACKEND_HOST environment variable can be used to control where upstream API requests are proxied to and defaults to our shared staging instance.
Unfortunately, we’ve lost the great developer experience afforded by hot module reload (HMR) because everything is inside the container. By adding a develop.watch section, we can preserve that:
Now, while working on the frontend, developers continue to benefit from the rapid iteration cycles due to HMR. Whenever a file is modified locally in the src/ directory, it’s synchronized into the container at /app/src.
If the package.json file is modified, the entire container is rebuilt, so that the RUN npm install step in the Dockerfile will be re-executed and install the latest dependencies. The best part is the only change to the workflow is running docker compose watch instead of npm start.
Much of this looks very similar to the frontend compose.yaml.
When files in the project directory change locally, they’re synchronized to /app inside the container, so the Flask dev server can handle hot reload. If the requirements.txt is changed, the entire container is rebuilt, so that the RUN pip install step in the Dockerfile will be re-executed and install the latest dependencies.
However, we’ve also added an include section that references the frontend project by its Git repository. The custom env_file points to a local path (in the backend repo), which sets BACKEND_HOST so that the frontend service container will proxy API requests to the backend service container instead of the default.
Note: Remote includes are an experimental feature. You’ll need to set COMPOSE_EXPERIMENTAL_GIT_REMOTE=1 in your environment to use Git references.
With this configuration, developers can now run the full stack while keeping the frontend and backend Compose projects independent and even in different Git repositories.
As developers, we’re used to sharing code library dependencies, and the include keyword brings this same reusability and convenience to your Compose development configurations.
What’s next?
There are still some rough edges. For example, the remote project is cloned to a temporary directory, which makes it impractical to use with watch mode when imported, as the files are not available for editing. Enabling bigger and more complex software projects to use Compose for flexible, personal environments is something we’re continuing to improve upon.
If you’re a Docker customer using Compose across microservices or repositories, we’d love to hear how we can better support you. Get in touch!
Docker Scout is a software supply chain solution designed to make it easier for developers to identify and fix supply chain issues before they hit production.
To do this, Docker Scout:
Gives developers a centralized view of the tools they already use to see all the critical information they need across the software supply chain
Makes clear recommendations on how to address those issues, including for security issues and opportunities to improve reliability efforts
Provides automation that highlights new defects, failures, or issues
Docker Scout allows you to prevent and address flaws where they start. By identifying issues earlier in the software development lifecycle and displaying information in Docker Desktop and the command line, Docker Scout reduces interruptions and rework.
Supply chain security is a big focus in software development, with attention from enterprises and governments. Software is complex, and when security, reliability, and stability issues arise, they’re often the result of an upstream library. So developers don’t just need to address issues in the software they write but also in the software their software uses.
These concerns apply just as much to open source projects as proprietary software. But the focus on improving the software supply chain results in an unfunded mandate for open source developers. A research study by the Linux Foundation found that almost 25% of respondents said the cost of security gaps was “high” or “very high.” Most open source projects don’t have the budget to address these gaps. With Docker Scout, we can reduce the burden on open source projects.
Conclusion
At Docker, we understand the importance of helping open source communities improve their software supply chain. We see this as a mutually beneficial relationship with the open source community. A well-managed supply chain doesn’t just help the projects that produce open source software; it helps downstream consumers through to the end user.
Join our “Improving Software Supply Chain Security for Open Source Projects” webinar on Wednesday, February 7, 2024 at 1 PM Eastern (1700 UTC). Watch on LinkedIn or on the Riverside streaming platform.
Learn how Docker Scout provides actionable insights into the software supply chain with real-time vulnerability identification, remediation recommendations, ...
Are you struggling to manage your data infrastructure on Kubernetes? Do you want a solution that is easy to use, powerful, and scalable? Then you might need KubeBlocks!
KubeBlocks is an open-source platform that makes it easy to deploy, manage, and scale databases, data warehouses, and other data-intensive applications on Kubernetes. It provides a rich set of features for monitoring, backup, and recovery, and it supports a wide variety of databases, including MySQL, PostgreSQL, MongoDB, Kafka, Redis, Vector databases, Pulsar, and more.
▬▬▬▬▬▬ Sponsorships ▬▬▬▬▬▬ If you are interested in sponsoring this channel, please use https://calendly.com/vfarcic/meet to book a timeslot that suits and we’ll go over the details. Or feel free to contact me over Twitter or LinkedIn (see below)
This post was contributed by Ethan Heilman, CTO at BastionZero.
OpenPubkey is the web’s new technology for adding public keys to standard single sign-on (SSO) interactions with identity providers that speak OpenID Connect (OIDC). OpenPubkey works by essentially turning an identity provider into a certificate authority (CA), which is a trusted entity that issues certificates that cryptographically bind an identity with a cryptographic public key. With OpenPubkey, any OIDC-speaking identity provider can bind public keys to identities today.
OpenPubkey is newly open-sourced through a collaboration of BastionZero, Docker, and the Linux Foundation. We’d love for you to try it out, contribute, and build your own use cases on it. You can check out the OpenPubkey repository on GitHub.
In this article, our goal is to show you how to use OpenPubkey to bind public keys to workload identities. We’ll concentrate on GitHub Actions workloads, because this is what is currently supported by the OpenPubkey open source project. We’ll also briefly cover how Docker is using OpenPubkey with GitHub Actions to sign in-toto attestations on Docker Official Images and improve supply chain security.
What’s an ID token?
Before we start, let’s review the OpenID Connect protocol. Identity providers that speak OIDC are usually called OpenID Providers, but we will just call them OPs in this article.
OIDC has an important artifact called an ID token. A user obtains an ID token after they complete their single sign-on to their OP. They can then present the ID token to a third-party service to prove that they have properly been authenticated by their OP.
The ID token includes the user’s identity (such as their email address) and is cryptographically signed by the OP. The third-party service can validate the ID token by querying the OP’s JSON Web Key Set (JWKS) endpoint, obtaining the OP’s public key, and then using the OP’s public key to validate the signature on the ID token. The OP’s public key is available by querying a JWKS endpoint hosted by the OP.
How do GitHub Actions obtain ID tokens?
So far, we’ve been talking about human identities (such as email addresses) and how they are used with ID tokens. But, our focus in this article is on workload identities. It turns out that Actions has a nice way to assign ID tokens to GitHub Actions.
Here’s how it works. GitHub runs an OpenID Provider. When a new GitHub Action is spun up, GitHub first assigns it a fresh API key and secret. The GitHub Action can then use its API key and secret to authenticate to GitHub’s OP. GitHub’s OP can validate this API key and secret (because it knows that it was assigned to the new GitHub Action) and then provide the GitHub Action with an OIDC ID token. This GitHub Action can now use this ID token to identify itself to third-party services.
When interacting with GitHub’s OP, Docker uses the job_workflow_ref claim in the ID token as the workflow’s “identity.” This claim identifies the location of the file that the GitHub Action is built from, so it allows the verifier to identify the file that generated the workflow and thus also understand and check the validity of the workflow itself. Here’s an example of how the claim could be set:
Other claims in the ID tokens issued by GitHub’s OP can be useful in other use cases. For example, there is a field called Actor or ActorID, whichis the identity of the person who kicked off the GitHub Action. This could be useful for checking that workload was kicked off by a specific person. (It’s less useful when the workload was started by an automated process.)
GitHub’s OP supports many other useful fields in the ID token. You can learn more about them in the GitHub OIDC documentation.
Creating a PK token for workloads
Now that we’ve seen how to identify workloads using GitHub’s OP, we will see how to bind that workload identity to its public key with OpenPubkey. OpenPubKey does this with a cryptographic object called the PK token.
To understand how this process works, let’s go back and look at how GitHub’s OP implements the OIDC protocol. The ID tokens generated by GitHub’s OP have a field called audience. Importantly, unlike user identity in workload identity, the audience field is chosen by the OIDC client that requests the ID token. When GitHub’s OP creates the ID token, it includes the audience along with the other fields (like job_workflow_ref and actor) that the OP signs when it creates the ID token.
So, in OpenPubkey, the GitHub Action workload runs an OpenPubkey client that first generates a new public-private key pair. Then, when the workload authenticates to GitHub’s OP with OIDC, it sets the audience field equal to the cryptographic hash of the workload’s public key along with some random noise.
Now, the ID token contains the GitHub OP’s signature on the workload’s identity (the job_workflow_ref field and other relevant fields) and on the hash of the workload’s public key. This is most of what we need to have GitHub’s OP bind the workload’s identity and public key.
In fact, the PK token is a JSON Web Signature (JWS) which roughlyconsists of:
The ID token, including the audience field, which contains a hash of the workload’s public key.
The workload’s public key.
The random noise used to compute the hash of the workload’s public key.
A signature, under the workload’s public key, of all the information in the PK token. (This signature acts as a cryptographic proof that the user has access to the user-held secret signing key that is certified in the PK token.)
The PK token can then be presented to any OpenPubkey verifier, which uses OIDC to obtain the GitHub OP’s public key from its JWKS end. The verifier then verifies the ID token using the GitHub OP public key and then verifies the rest of the other fields in the PK token using the workload’s public key. Now the verifier knows the public key of the workload (as identified by its job_workflow_ref or other fields in the ID token) and can use this public key for whatever cryptography it wants to do.
Can you use ephemeral keys with OpenPubkey?
Yes! An ephemeral key is a key that is only used for a short period of time. Ephemeral keys are nice because there is no need for long-term management of the private key; it can be deleted when it is no longer needed, which improves security and reduces operational overhead.
Here’s how to do this with OpenPubkey. You choose a public-private key pair, authenticate to the OP to obtain a PK token for the public key, sign your object using the private key, and finally throw away the private key.
One-time-use PK token
We can take this a step further and ensure the PK token may only be associated with a single signed object. Here’s how it works. To start, we take a hash of the object to be signed. Then, when the workload authenticates to GitHub’s OP, set the audience claim to equal to the cryptographic hash of the following items:
The public key
The hash of the object to be signed
Some random noise
Finally, OpenPubkey verifier obtains the signed object and its one-time-use PK token, and then validates the PK token by additionally checking that the hash of the signed object is included in the audience claim. Now, you have a one-time-use PK token. You can learn more about this feature of OpenPubkey in the repo.
How will Docker use OpenPubkey to sign Docker Official Images?
Docker will be using OpenPubkey with GitHub Actions workloads to sign in-toto attestations on Docker Official Images. Docker Official Images will be created using a GitHub Action workload. The workload creates a fresh ephemeral public-private key pair, obtains the PK token for the public key via OpenPubkey, and finally signs attestations on the image using the private key.
The private key is then deleted, and the image, its signature, and the PK token will be made available on the Docker Hub container registry. This approach is nice because it doesn’t require the signer to maintain or store the private key.
What else can you do with OpenPubkey and GitHub Actions workloads?
Check out the following ideas on how to put OpenPubkey and GitHub Actions to work for you.
Signing private artifacts with a one-time key
Consider signing artifacts that will be stored in a private repository. You can use OpenPubkey if you want to have a GitHub Action cryptographically sign an artifact using a one-time-use key. A nice thing about this approach is that it doesn’t require you to expose information in a public repository or transparency log. Instead, you need to post the artifact, its signature, and its PK token in the private repository. This capability is useful for private code repositories or internal build systems where you don’t want to reveal to the world what is being built, by whom, when, or how frequently.
If relevant, you could also consider using the actor and actor-ID claim to bind the human who builds a particular artifact to the signed artifact itself.
Authenticating workload-to-workload communication
Suppose you want one workload (call it Bob) to process an artifact created by another workload (call it Alice). If the Alice workload is a GitHub Action, the artifact it creates could be signed using OpenPubkey and passed on to the Bob workload, which uses an OpenPubkey verifier to verify it using the GitHub OP’s public key (which it would obtain from the GitHub OP’s JWKS url). This approach might be useful in a multi-stage CI/CD process.
And other things, too!
These are just strawman ideas. The whole point of this post is for you to try out OpenPubkey, contribute, and build your own use cases on it.
Other technical issues we need to think about
Before we wrap up, we need to discuss a few technical questions.
Aren’t ID tokens supposed to remain private?
You might worry about applications of OpenPubkey where the ID token is broadly exposed to the public inside the PK token. For example, in Docker Official Image signing use case, the PK tokens are made available to the public in the Docker Hub container registry. If the ID token is broadly exposed to the public, there is a risk that the ID token could be replayed and used for unauthorized access to other services.
For this reason, we have a slightly different PK token for applications where the PK token is made broadly available to the public.
For those applications, OpenPubkey strips the OP’s signature from the ID token before including the PK token. The OP’s signature is replaced with a Guillou-Quisquater (GQ) non-interactive proof-of-knowledge for an RSA signature (which is also known as a “GQ signature”). Now, the ID Token cannot be replayed against other services because the OP’s signature is removed, but the security of the OP’s RSA signature is maintained by the GQ Signature.
So, in applications where the PK token must be broadly exposed to the public, the PK token is a JSON Web Signature, which consists of:
The ID token excluding the OP’s signature
A GQ signature on the ID token
The user’s public key
The random noise used to compute the hash of the user’s public key
A signature, under the user’s public key, of all the information in the PK token
The GQ signature allows the client to prove that the ID token was validly signed by the OP, without revealing the OP’s signature. The OpenPubkey client generates the GQ signature to cryptographically prove that the client knows the OP’s signature on the ID token, while still keeping the OP’s signature secret. GQ signatures only work with RSA, but this is fine because every OpenID Connect provider is required to support RSA.
Because GQ signatures are larger and slower than regular signatures, we recommend using them only for use cases where the PK token must be made broadly available to the public. BastionZero’s infrastructure access use case does not use GQ signatures because it does not require the PK token to be made public. Instead, the user only exposes their PK token to the target (e.g., server, container, cluster, database) that they want to access; this is the same way an ID token is usually exposed with OpenID Connect.
GQ signatures might not be necessary when authenticating workload-to-workload communications; if the Alice workload is passing the signed artifact and its PK token to the Bob workload only, there is less of a concern as the PK token is broadly available to the public.
What happens when the OP rotates its OpenID Connect key?
OPs have OpenID Connect signing keys that change over time (e.g., every two weeks). What happens if we need to use a PK token after the OP rotates OpenID Connect key that signed the PK Token?
For some use cases, the lifetime of a PK token is typically short. With BastionZero’s infrastructure access use case, for instance, a PK token will not be used for longer than 24 hours. In this use case, these timing problems are solved by (1) having user re-authenticate to the IdP and create a new PK token whenever the IdP rotates it’s key, and (2) having the OpenPubkey verifier check that the client also has a valid OIDC Refresh token along with the PK token whenever the ID token expires.
For some use cases, the PK token has a long life, so we do need to worry about OP rotating their OpenID Connect keys. With Docker’s attestation signing use case, this problem is solved by having TUF additionally store a historical log of the OP’s signing key. Anyone can keep a historical log of the OP public keys for use after they expire. In fact, we envision a future where OP’s might keep this historical log themselves.
That’s it for now! You can check out the OpenPubkey repo on GitHub. We’d love for you to join the project, contribute, and identify other use cases where OpenPubkey might be useful.
We’re shifting testing “left” to help developers ship quality apps faster
I’m thrilled to announce that Docker is whale-coming AtomicJar, the makers of Testcontainers, to the Docker family. With its support for Java, .NET, Go, Node.js, and six other programming languages, together with its container-based testing automation, Testcontainers has become the de facto standard test framework for the developer’s ”inner loop.” Why? The results speak for themselves — Testcontainers enables step-function improvements in both the quality and speed of application delivery.
This addition continues Docker’s focus on improving the developer experience to maximize the time developers spend building innovative apps. Docker already accelerates the “inner loop” app development steps — build, verify (through Docker Scout), run, debug, and share — and now, with AtomicJar and Testcontainers, we’re adding “test.” As a result, developers using Docker will be able to deliver quality applications with less effort, even faster than before.
Testcontainers itself is a great open source success story in the developer tools ecosystem. Last year, Testcontainers saw a 100% increase in Docker Hub pulls, from 50 million to 100 million, making it one of the fastest-growing Docker Hub projects. Furthermore, Testcontainers has transformed testing at organizations like DoorDash, Netflix, Spotify, and Uber and thousands more.
One of the more exciting things about whale-coming AtomicJar is the bringing together our open source communities. Specifically, the Testcontainers community has deep roots in the programming language communities above. We look forward to continuing to support the Testcontainers open source project and look forward to what our teams do to expand it further.
Please join me in whale-coming AtomicJar and Testcontainers to Docker!
sj
FAQ | Docker Acquisition of AtomicJar
With Docker’s acquisition of AtomicJar and associated Testcontainers projects, you’re sure to have questions. We’ve answered the most common ones in this FAQ.
As with all of our open source efforts, Docker strives to do right by the community. We want this acquisition to benefit everyone — community and customer — in keeping with our developer obsession.
What will happen to Testcontainers Cloud customers? Customers of AtomicJar’s paid offering, Testcontainers Cloud, will continue while we work to develop new and better integration options. Existing Testcontainers Cloud subscribers will see an update to the supplier on their invoices, but no other billing changes will occur.
Will Testcontainers become closed-source? There are no plans to change the licensing structure of Testcontainers’s open source components. Docker has always valued the contributions of open source communities.
Will Testcontainers or its companion projects be discontinued? There are no plans to discontinue any Testcontainers projects.
Will people still be able to contribute to Testcontainers’s open source projects? Yes! Testcontainers has always benefited from outside collaboration in the form of feedback, discussion, and code contributions, and there’s no desire to change that relationship. For more information about how to participate in Testcontainers’s development, see the contributing guidelines for Java, Go, and .NET.
What about other downstream users, companies, and projects using Testcontainers? Testcontainers’ open source licenses will continue to allow the embedding and use of Testcontainers by other projects, products, and tooling.
Who will provide support for Testcontainers projects and products? In the short term, support for Testcontainers’s projects and products will continue to be provided through the existing support channels. We will work to merge support into Docker’s channels in the near future.
How can I get started with Testcontainers? To get started with Testcontainers follow this guide or one of the guides for a language of your choice:
Open source software – any software that is freely and available shared with others – has quickly become a staple in the world of software development. Back in 1998, only around 10% of businesses would use open source software, preferring to develop proprietary software due to security concerns. A decade later, in 2011, and this […]
Vector databases have been getting a lot of attention since the developer community realized how they can enhance large language models (LLMs). Weaviate is an open source vector database that enables modern search capabilities, such as vector search, hybrid search, and generative search. With Weaviate, you can build advanced LLM applications, next-level search systems, recommendation systems, and more.
This article explains what vector databases are and highlights key features of the Weaviate vector database. Learn how to install Weaviate on Docker using Docker Compose so you can take advantage of semantic search within your Dockerized environment.
Introducing the Weaviate vector database
The core feature of vector databases is storing vector embeddings of data objects. This functionality is especially helpful with the growing amount of unstructured data (e.g., text or images), which is difficult to manage and process with traditional relational databases. The vector embeddings are a numerical representation of the data objects — usually generated by a machine learning (ML) model — and enable the search and retrieval of data based on semantic similarity (vector search).
Vector databases do much more than just store vector embeddings: As you can imagine, retrieving data based on similarity requires a lot of comparing between objects and thus can take a long time. In contrast to other types of databases that can store vector embeddings, a vector database can retrieve data fast. To enable low-latency search queries, vector databases use specific algorithms to index the data.
Additionally, some vector databases, like Weaviate, store the vector embeddings and the original data object, which lets you combine traditional search with modern vector search for more accurate search results.
With these functionalities, vector databases are usually used in search or similar tasks (e.g., recommender systems). With the recent advancements in the LLM space, however, vector databases have also proven effective at providing long-term memory and domain-specific context to conversational LLMs. This means that you can leverage LLM capabilities on your private data or your specific field of expertise.
Key highlights of the Weaviate vector database include:
Open source: Weaviate is open source and available for anybody to use wherever they want. It is also available as a managed service with SaaS and hybrid SaaS options.
Horizontal scalability: You can scale seamlessly into billions of data objects for your exact needs, such as maximum ingestion, largest possible dataset size, maximum queries per second, etc.
Lightning-fast vector search: You can perform lightning-fast pure vector similarity search over raw vectors or data objects, even with filters. Weaviate typically performs nearest-neighbor searches of millions of objects in considerably less than 100ms (see our benchmark).
Combined keyword and vector search (hybrid search): You can store both data objects and vector embeddings. This approach allows you to combine keyword-based and vector searches for state-of-the-art search results.
Optimized for cloud-native environments: Weaviate has the fault tolerance of a cloud-native database, and the core Docker image is comparably small at 18 MB.
Modular ecosystem for seamless integrations: You can use Weaviate standalone (aka “bring your own vectors”) or with various optional modules that integrate directly with OpenAI, Cohere, Hugging Face, etc., to enable easy use of state-of-the-art ML models. These modules can be used as vectorizers to automatically vectorize any media type (text, images, etc.) or as generative modules to extend Weaviate’s core capabilities (e.g., question answering, generative search, etc.).
Prerequisites
Ensure you have both the docker and the docker-compose CLI tools installed. For the following section, we assume you have Docker 17.09.0 or higher and Docker Compose V2 installed. If your system has Docker Compose V1 installed instead of V2, use docker-compose instead of docker compose. You can check your Docker Compose version with:
$ docker compose version
How to Configure the Docker Compose File for Weaviate
To start Weaviate with Docker Compose, you need a Docker Compose configuration file, typically called docker-compose.yml. Usually, there’s no need to obtain individual images, as we distribute entire Docker Compose files.
You can obtain a Docker Compose file for Weaviate in two different ways:
Docker Compose configurator on the Weaviate website (recommended): The configurator allows you to customize your docker-compose.yml file for your purposes (including all module containers) and directly download it.
Manually: Alternatively, if you don’t want to use the configurator, copy and paste one of the example files from the documentation and manually modify it.
Setting up a persistent volume to avoid data loss when you restart the container and improve reading and writing speeds is recommended.
You can set a persistent volume in two ways:
With a named volume: Docker will create a named volume weaviate_data and mount it to the PERSISTENCE_DATA_PATH inside the container after starting Weaviate with Docker Compose:
services:
weaviate:
volumes:
- weaviate_data:/var/lib/weaviate
# etc.
volumes:
weaviate_data:
With host binding: Docker will mount ./weaviate_data on the host to the PERSISTENCE_DATA_PATH inside the container after starting Weaviate with Docker Compose:
services:
weaviate:
volumes:
- ./weaviate_data:/var/lib/weaviate
# etc.
Step 3: Modules
Weaviate can be used with various modules, which integrate directly with inferencing services like OpenAI, Cohere, or Hugging Face. These modules can be used to vectorize any media type at import and search time automatically or to extend Weaviate’s core capabilities with generative modules.
You can also use Weaviate without any modules (standalone). In this case, no model inference is performed at import or search time, meaning you need to provide your own vectors in both scenarios. If you don’t need any modules, you can skip to Step 4: Runtime.
Configure modules for Docker Compose file (Figure 3):
Figure 3: The Weaviate Docker Compose configurator step to define if modules will be used, or if running standalone is desired.
Currently, Weaviate integrates three categories of modules:
Retriever and vectorizer modules automatically vectorize any media type (text, images, etc.) at import and search time. There are also re-ranker modules available for re-ranking search results.
Reader and generator modules can be used to extend Weaviate’s core capabilities after retrieving the data for generative search, question answering, named entity recognition (NER), and summarization.
Other modules are available for spell checking or for enabling using your custom modules.
Note that many modules (e.g., transformer models) are neural networks built to run on GPUs. Although you can run them on CPU, enabling GPU `ENABLE_CUDA=1`, if available, will result in faster inference.
In the final step of the configurator, select Docker Compose for your runtime (Figure 4):
Figure 4: The final step of the Weaviate Docker Compose configurator where “Docker Compose” can be selected as the runtime.
Step 5: Download and further customization
Once your configuration is complete, you will see a snippet similar to the following to download the docker-compose.yml file, which has been adjusted to your selected configuration.
After downloading the Docker Compose file from the configurator, you can directly start Weaviate on Docker or customize it further.
You can set additional environment variables to further customize your Weaviate setup (e.g., by defining authentication and authorization). Additionally, you can create a multi-node setup with Weaviate by defining a founding member and other members in the cluster.
Founding member: Set up one node as a “founding” member by configuring CLUSTER_GOSSIP_BIND_PORT and CLUSTER_DATA_BIND_PORT:
weaviate-node-1: # Founding member service name
... # truncated for brevity
environment:
CLUSTER_HOSTNAME: 'node1'
CLUSTER_GOSSIP_BIND_PORT: '7100'
CLUSTER_DATA_BIND_PORT: '7101'
Other members in the cluster: For each further node, configure CLUSTER_GOSSIP_BIND_PORT and CLUSTER_DATA_BIND_PORT and configure to join the founding member’s cluster using the CLUSTER_JOIN variable:
weaviate-node-2:
... # truncated for brevity
environment:
CLUSTER_HOSTNAME: 'node2'
CLUSTER_GOSSIP_BIND_PORT: '7102'
CLUSTER_DATA_BIND_PORT: '7103'
CLUSTER_JOIN: 'weaviate-node-1:7100' # This must be the service name of the "founding" member node.
Optionally, you can set a hostname for each node using CLUSTER_HOSTNAME.
Note that it’s a Weaviate convention to set the CLUSTER_DATA_BIND_PORT to 1 higher than CLUSTER_GOSSIP_BIND_PORT.
How to run Weaviate on Docker
Once you have your Docker Compose file configured to your needs, you can run Weaviate in your Docker environment.
Start Weaviate
Before starting Weaviate on Docker, ensure that the Docker Compose file is named exactly docker-compose.yml and that you are in the same folder as the Docker Compose file.
Then, you can start up with the whole setup by running:
$ docker compose up -d
The -d option runs containers in detached mode. This means that your terminal will not attach to the log outputs of all the containers.
If you want to attach to the logs of specific containers (e.g., Weaviate), you can run the following command:
Congratulations! Weaviate is now running and is ready to be used.
Stop Weaviate
To avoid losing your data, shut down Weaviate with the following command:
$ docker compose down
This will write all the files from memory to disk.
Conclusion
This article introduced vector databases and how they can enhance LLM applications. Specifically, we highlighted the open source vector database Weaviate, whose advantages include fast vector search at scale, hybrid search, and integration modules to state-of-the-art ML models from OpenAI, Cohere, Hugging Face, etc.
We also provided a step-by-step guide on how to install Weaviate on Docker using Docker Compose, noting that you can obtain a docker-compose.yml file from the Weaviate Docker Compose configurator, which helps you to customize your Docker Compose file for your specific needs.
Visit our AI/ML page and read the article collection to learn more about how developers are using Docker to accelerate the development of their AI/ML applications.
Deep learning has revolutionized the field of artificial intelligence (AI) by enabling machines to learn and generate content that mimics human-like creativity. One advancement in this domain is Stable Diffusion, a text-to-image model released in 2022.
Stable Diffusion has gained significant attention for its ability to generate highly detailed images conditioned on text descriptions, thereby opening up new possibilities in areas such as creative design, visual storytelling, and content generation. With its open source nature and accessibility, Stable Diffusion has become a go-to tool for many researchers and developers seeking to harness the power of deep learning.
In this article, we will explore how to optimize deep learning workflows by leveraging Stable Diffusion alongside Docker on WSL 2, enabling seamless and efficient experimentation with this cutting-edge technology.
In this comprehensive guide, we will walk through the process of setting up the Stable Diffusion WebUI Docker, which includes enabling WSL 2 and installing Docker Desktop. You will learn how to download the required code from GitHub and initialize it using Docker Compose.
The guide provides instructions on adding additional models and managing the system, covering essential tasks such as reloading the UI and determining the ideal location for saving image output. Troubleshooting steps and tips for monitoring hardware and GPU usage are also included, ensuring a smooth and efficient experience with Stable Diffusion WebUI (Figure 1).
Figure 1: Stable Diffusion WebUI.
Why use Docker Desktop for Stable Diffusion?
In the realm of image-based generative AI, setting up an effective execution and development environment on a Windows PC can present particular challenges. These challenges arise due to differences in software dependencies, compatibility issues, and the need for specialized tools and frameworks. Docker Desktop emerges as a powerful solution to tackle these challenges by providing a containerization platform that ensures consistency and reproducibility across different systems.
By leveraging Docker Desktop, we can create an isolated environment that encapsulates all the necessary components and dependencies required for image-based generative AI workflows. This approach eliminates the complexities associated with manual software installations, conflicting library versions, and system-specific configurations.
Using Stable Diffusion WebUI
The Stable Diffusion WebUI is a browser interface that is built upon the Gradio library, offering a convenient way to interact with and explore the capabilities of Stable Diffusion. Gradio is a powerful Python library that simplifies the process of creating interactive interfaces for machine learning models.
Setting up the Stable Diffusion WebUI environment can be a tedious and time-consuming process, requiring multiple steps for environment construction. However, a convenient solution is available in the form of Stable Diffusion WebUI Docker project. This Docker image eliminates the need for manual setup by providing a preconfigured environment.
If you’re using Windows and have Docker Desktop installed, you can effortlessly build and run the environment using the docker-compose command. You don’t have to worry about preparing libraries or dependencies beforehand because everything is encapsulated within the container.
You might wonder whether there are any problems because it’s a container. I was anxious before I started using it, but I haven’t had any particular problems so far. The images, models, variational autoencoders (VAEs), and other data that are generated are shared (bind mounted) with my Windows machine, so I can exchange files simply by dragging them in Explorer or in the Files of the target container on Docker Desktop.
The most trouble I had was when I disabled the extension without backing it up, and in a moment blew away about 50GB of data that I had spent half a day training. (This is a joke!)
Architecture
I’ve compiled a relatively simple procedure to start with Stable Diffusion using Docker Desktop on Windows.
Prerequisites:
Windows 10 Pro, 21H2 Build 19044.2846
16GB RAM
NVIDIA GeForce RTX 2060 SUPER
WSL 2 (Ubuntu)
Docker Desktop 4.18.0 (104112)
Setup with Docker Compose
We will use the WebUI called AUTOMATIC1111 to utilize Stable Diffusion this time. The environment for these will be constructed using Docker Compose. The main components are shown in Figure 2.
Figure 2: Configuration built using Docker Compose.
The configuration of Docker Compose is defined in docker-compose.yml. We are using a Compose extension called x-base_service to describe the major components common to each service.
To start, there are settings for bind mount between the host and the container, including /data, which loads modes, and /output, which outputs images. Then, we make the container recognize the GPU by loading the NVIDIA driver.
Furthermore, the service named sd-auto:58 runs AUTOMATIC1111, WebUI for Stable Diffusion, within the container. Because there is a port mapping (TCP:7860), between the host and the container in the aforementioned common service settings, it is possible to access from the browser on the host side to the inside of the container.
Getting Started
Prerequisite
WSL 2 must be activated and Docker Desktop installed.
On the first execution, it downloads 12GB of Stable Diffusion 1.5 models, etc. The Web UI cannot be used until this download is complete. Depending on your connection, it may take a long time until the first startup.
Downloading the code
First, download the Stable Diffusion WebUI Docker code from GitHub. If you download it as a ZIP, click Code > Download ZIP and the stable-diffusion-webui-docker-master.zip file will be downloaded (Figure 3).
Unzip the file in a convenient location. When you expand it, you will find a folder named stable-diffusion-webui-docker-master. Open the command line or similar and run the docker compose command inside it.
Figure 3: Downloading the configuration for Docker Compose from the repository.
Or, if you have an environment where you can use Git, such as Git for Windows, it’s quicker to download it as follows:
In this case, the folder name is stable-diffusion-webui-docker. Move it with cd stable-diffusion-webui-docker.
Supplementary information for those familiar with Docker
If you just want to get started, you can skip this section.
By default, the timezone is UTC. To adjust the time displayed in the log and the date of the directory generated under output/txt2img to Japan time, add TZ=Asia/Tokyo to the environment variables of the auto service. Specifically, add the following description to environment:.
The rest of the process is as described in the GitHub documentation. Inside the folder where the code is expanded, run the following command:
docker compose --profile download up --build
After the command runs, the log of a container named webui-docker-download-1 will be displayed on the screen. For a while, the download will run as follows, so wait until it is complete:
If a code other than 0 comes out like the following, the download process has failed:
webui-docker-download-1 | 42c377|OK | 426KiB/s|/data/StableDiffusion/sd-v1-5-inpainting.ckpt
webui-docker-download-1 |
webui-docker-download-1 | Status Legend:
webui-docker-download-1 | (OK):download completed.(ERR):error occurred.
webui-docker-download-1 |
webui-docker-download-1 | aria2 will resume download if the transfer is restarted.
webui-docker-download-1 | If there are any errors, then see the log file. See '-l' option in help/m
an page for details.
webui-docker-download-1 exited with code 24
In this case, run the command again and check whether it ends successfully. Once it finishes successfully, run the command to start the WebUI.
Note: The following is for AUTOMATIC1111’s UI and GPU specification:
docker compose --profile auto up --build
When you run the command, loading the model at the first startup may take a few minutes. It may look like it’s frozen like the following display, but that’s okay:
webui-docker-auto-1 | LatentDiffusion: Running in eps-prediction mode
webui-docker-auto-1 | DiffusionWrapper has 859.52 M params.
If you wait for a while, the log will flow, and the following URL will be displayed:
webui-docker-auto-1 | Running on local URL: http://0.0.0.0:7860
Now the startup preparation of the Web UI is set. If you open http://127.0.0.1:7860 from the browser, you can see the Web UI. Once open, select an appropriate model from the top left of the screen, write some text in the text field, and select the Generate button to start generating images (Figure 4).
Figure 4: After selecting the model, input the prompt and generate the image.
When you click, the button will be reversed. Wait until the process is finished (Figure 5).
Figure 5: Waiting until the image is generated.
At this time, the log of image generation appears on the terminal you are operating, and you can also check the similar display by looking at the log of the container on Docker Desktop (Figure 6).
Figure 6: 100% indicates that the image generation is complete.
When the status reaches 100%, the generation of the image is finished, and you can check it on the screen (Figure 7).
Figure 7: After inputting “Space Cat” in the prompt, a cat image was generated at the bottom right of the screen.
The created images are automatically saved in the output/txt2img/date folder directly under the directory where you ran the docker compose command.
To stop the launched WebUI, enter Ctrl+C on the terminal that is still running the docker compose command.
Gracefully stopping... (press Ctrl+C again to force)
Aborting on container exit...
[+] Running 1/1
? Container webui-docker-auto-1 Stopped 11.4s
canceled
When the process ends successfully, you will be able to run the command again. To use the WebUI again after restarting, re-run the docker compose command:
docker compose --profile auto up --build
To see the operating hardware status, use the task manager to look at the GPU status (Figure 8).
Figure 8: From the Performance tab of the Windows Task Manager, you can monitor the processing of CUDA and similar tasks on the GPU.
To check whether the GPU is visible from inside the container and to see whether the information comes out, run the nvidia-smi command from docker exec or the Docker Desktop terminal.
root@e37fcc5a5810:/stable-diffusion-webui# nvidia-smi
Mon Apr 17 07:42:27 2023
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 530.41.03 Driver Version: 531.41 CUDA Version: 12.1 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 NVIDIA GeForce RTX 2060 S... On | 00000000:01:00.0 On | N/A |
| 42% 40C P8 6W / 175W| 2558MiB / 8192MiB | 2% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| 0 N/A N/A 149 C /python3.10 N/A |
+---------------------------------------------------------------------------------------+
Adding models and VAEs
If you download a model that is not included from the beginning, place files with extensions, such as .safetensors in stable-diffusion-webui-docker\data\StableDiffusion. In the case of VAE, place .skpt files in stable-diffusion-webui-docker\data\VAE.
If you’re using Docker Desktop, you can view and operate inside on the Files of the webui-docker-auto-1 container, so you can also drag it into Docker Desktop.
Figure 9 shows the Docker Desktop screen. It says MOUNT in the Note column, and it shares the information in the folder with the container from the Windows host side.
Figure 9: From the Note column, you can see whether the folder is mounted or has been modified.
Now, after placing the file, a link to Reload UI is shown in the footer of the WebUI, so select there (Figure 10).
Figure 10: By clicking Reload UI, the WebUI settings are reloaded.
When you select Reload UI, the system will show a loading screen, and the browser connection will be cut off. When you reload the browser, the model and VAE files are automatically loaded. To remove a model, delete the model file from data\StableDiffusion.
Conclusion
With Docker Desktop, image generation using the latest generative AI environment can be done easier than ever. Typically, a lot of time and effort is required just to set up the environment, but Docker Desktop solves this complexity. If you’re interested, why not take a challenge in the world of generative AI? Enjoy!
In today’s fast-paced digital era, conversational AI chatbots have emerged as a game-changer in delivering efficient and personalized user interactions. These artificially intelligent virtual assistants are designed to mimic human conversations, providing users with quick and relevant responses to queries.
A crucial aspect of building successful chatbots is the ability to handle frequently asked questions (FAQs) seamlessly. FAQs form a significant portion of user queries in various domains, such as customer support, e-commerce, and information retrieval. Being able to provide accurate and prompt answers to common questions not only improves user satisfaction but also frees up human agents to focus on more complex tasks.
In this article, we’ll look at how to use the open source Rasa framework along with Docker to build and deploy a containerized, conversational AI chatbot.
Meet Rasa
To tackle the challenge of FAQ handling, developers turn to sophisticated technologies like Rasa, an open source conversational AI framework. Rasa offers a comprehensive set of tools and libraries that empower developers to create intelligent chatbots with natural language understanding (NLU) capabilities. With Rasa, you can build chatbots that understand user intents, extract relevant information, and provide contextual responses based on the conversation flow.
Rasa allows developers to build and deploy conversational AI chatbots and provides a flexible architecture and powerful NLU capabilities (Figure 1).
Figure 1: Overview of Rasa.
Rasa is a popular choice for building conversational AI applications, including chatbots and virtual assistants, for several reasons. For example, Rasa is an open source framework, which means it is freely available for developers to use, modify, and contribute to. It provides a flexible and customizable architecture that gives developers full control over the chatbot’s behavior and capabilities.
Rasa’s NLU capabilities allow you to extract intent and entity information from user messages, enabling the chatbot to understand and respond appropriately. Rasa supports different language models and machine learning (ML) algorithms for accurate and context-aware language understanding.
Rasa also incorporates ML techniques to train and improve the chatbot’s performance over time. You can train the model using your own training data and refine it through iterative feedback loops, resulting in a chatbot that becomes more accurate and effective with each interaction.
Additionally, Rasa can scale to handle large volumes of conversations and can be extended with custom actions, APIs, and external services. This capability allows you to integrate additional functionalities, such as database access, external API calls, and business logic, into your chatbot.
Why containerizing Rasa is important
Containerizing Rasa brings several important benefits to the development and deployment process of conversational AI chatbots. Here are four key reasons why containerizing Rasa is important:
1. Docker provides a consistent and portable environment for running applications.
By containerizing Rasa, you can package the chatbot application, its dependencies, and runtime environment into a self-contained unit. This approach allows you to deploy the containerized Rasa chatbot across different environments, such as development machines, staging servers, and production clusters, with minimal configuration or compatibility issues.
Docker simplifies the management of dependencies for the Rasa chatbot. By encapsulating all the required libraries, packages, and configurations within the container, you can avoid conflicts with other system dependencies and ensure that the chatbot has access to the specific versions of libraries it needs. This containerization eliminates the need for manual installation and configuration of dependencies on different systems, making the deployment process more streamlined and reliable.
2. Docker ensures the reproducibility of your Rasa chatbot’s environment.
By defining the exact dependencies, libraries, and configurations within the container, you can guarantee that the chatbot will run consistently across different deployments.
3. Docker enables seamless scalability of the Rasa chatbot.
With containers, you can easily replicate and distribute instances of the chatbot across multiple nodes or servers, allowing you to handle high volumes of user interactions.
4. Docker provides isolation between the chatbot and the host system and between different containers running on the same host.
This isolation ensures that the chatbot’s dependencies and runtime environment do not interfere with the host system or other applications. It also allows for easy management of dependencies and versioning, preventing conflicts and ensuring a clean and isolated environment in which the chatbot can operate.
Building an ML FAQ model demo application
By combining the power of Rasa and Docker, developers can create an ML FAQ model demo that excels in handling frequently asked questions. The demo can be trained on a dataset of common queries and their corresponding answers, allowing the chatbot to understand and respond to similar questions with high accuracy.
In this tutorial, you’ll learn how to build an ML FAQ model demo using Rasa and Docker. You’ll set up a development environment to train the model and then deploy the model using Docker. You will also see how to integrate a WebChat UI frontend for easy user interaction. Let’s jump in.
Getting started
The following key components are essential to completing this walkthrough:
This file describes the chatbot’s domain and includes crucial data including intents, entities, actions, and answers. It outlines the conversation’s structure, including the user’s input, and the templates for the bot’s responses.
version: "3.1"
intents:
- greet
- goodbye
- affirm
- deny
- mood_great
- mood_unhappy
- bot_challenge
responses:
utter_greet:
- text: "Hey! How are you?"
utter_cheer_up:
- text: "Here is something to cheer you up:"
image: "https://i.imgur.com/nGF1K8f.jpg"
utter_did_that_help:
- text: "Did that help you?"
utter_happy:
- text: "Great, carry on!"
utter_goodbye:
- text: "Bye"
utter_iamabot:
- text: "I am a bot, powered by Rasa."
session_config:
session_expiration_time: 60
carry_over_slots_to_new_session: true
As shown previously, this configuration file includes intents, which represent the different types of user inputs the bot can understand. It also includes responses, which are the bot’s predefined messages for various situations. For example, the bot can greet the user, provide a cheer-up image, ask if the previous response helped, express happiness, say goodbye, or mention that it’s a bot powered by Rasa.
The session configuration sets the expiration time for a session (in this case, 60 seconds) and specifies whether the bot should carry over slots (data) from a previous session to a new session.
The NLU training data are defined in this file. It includes input samples and the intents and entities that go with them. The NLU model, which connects user inputs to the right actions, is trained using this data.
version: "3.1"
nlu:
- intent: greet
examples: |
- hey
- hello
- hi
- hello there
- good morning
- good evening
- moin
- hey there
- let's go
- hey dude
- goodmorning
- goodevening
- good afternoon
- intent: goodbye
examples: |
- cu
- good by
- cee you later
- good night
- bye
- goodbye
- have a nice day
- see you around
- bye bye
- see you later
- intent: affirm
examples: |
- yes
- y
- indeed
- of course
- that sounds good
- correct
- intent: deny
examples: |
- no
- n
- never
- I don't think so
- don't like that
- no way
- not really
- intent: mood_great
examples: |
- perfect
- great
- amazing
- feeling like a king
- wonderful
- I am feeling very good
- I am great
- I am amazing
- I am going to save the world
- super stoked
- extremely good
- so so perfect
- so good
- so perfect
- intent: mood_unhappy
examples: |
- my day was horrible
- I am sad
- I don't feel very well
- I am disappointed
- super sad
- I'm so sad
- sad
- very sad
- unhappy
- not good
- not very good
- extremly sad
- so saad
- so sad
- intent: bot_challenge
examples: |
- are you a bot?
- are you a human?
- am I talking to a bot?
- am I talking to a human?
This configuration file defines several intents, which represent different types of user inputs that the chatbot can recognize. Each intent has a list of examples, which are example phrases or sentences that users might type or say to express that particular intent.
You can customize and expand upon this configuration by adding more intents and examples that are relevant to your chatbot’s domain and use cases.
This file is used to define the training stories, which serve as examples of user-chatbot interactions. A series of user inputs, bot answers, and the accompanying intents and entities make up each story.
The stories.yml file you provided contains a few training stories for a Rasa chatbot. These stories represent different conversation paths between the user and the chatbot. Each story consists of a series of steps, where each step corresponds to an intent or an action.
Here’s a breakdown of steps for the training stories in the file:
Story: happy path
User greets with an intent: greet
Bot responds with an action: utter_greet
User expresses a positive mood with an intent: mood_great
Bot acknowledges the positive mood with an action: utter_happy
Story: sad path 1
User greets with an intent: greet
Bot responds with an action: utter_greet
User expresses an unhappy mood with an intent: mood_unhappy
Bot tries to cheer the user up with an action: utter_cheer_up
Bot asks if the previous response helped with an action: utter_did_that_help
User confirms that it helped with an intent: affirm
Bot acknowledges the confirmation with an action: utter_happy
Story: sad path 2
User greets with an intent: greet
Bot responds with an action: utter_greet
User expresses an unhappy mood with an intent: mood_unhappy
Bot tries to cheer the user up with an action: utter_cheer_up
Bot asks if the previous response helped with an action: utter_did_that_help
User denies that it helped with an intent: deny
Bot says goodbye with an action: utter_goodbye
These training stories are used to train the Rasa chatbot on different conversation paths and to teach it how to respond appropriately to user inputs based on their intents.
The configuration parameters for your Rasa project are contained in this file.
# The config recipe.
# https://rasa.com/docs/rasa/model-configuration/
recipe: default.v1
# The assistant project unique identifier
# This default value must be replaced with a unique assistant name within your deployment
assistant_id: placeholder_default
# Configuration for Rasa NLU.
# https://rasa.com/docs/rasa/nlu/components/
language: en
pipeline:
# # No configuration for the NLU pipeline was provided. The following default pipeline was used to train your model.
# # If you'd like to customize it, uncomment and adjust the pipeline.
# # See https://rasa.com/docs/rasa/tuning-your-model for more information.
# - name: WhitespaceTokenizer
# - name: RegexFeaturizer
# - name: LexicalSyntacticFeaturizer
# - name: CountVectorsFeaturizer
# - name: CountVectorsFeaturizer
# analyzer: char_wb
# min_ngram: 1
# max_ngram: 4
# - name: DIETClassifier
# epochs: 100
# constrain_similarities: true
# - name: EntitySynonymMapper
# - name: ResponseSelector
# epochs: 100
# constrain_similarities: true
# - name: FallbackClassifier
# threshold: 0.3
# ambiguity_threshold: 0.1
# Configuration for Rasa Core.
# https://rasa.com/docs/rasa/core/policies/
policies:
# # No configuration for policies was provided. The following default policies were used to train your model.
# # If you'd like to customize them, uncomment and adjust the policies.
# # See https://rasa.com/docs/rasa/policies for more information.
# - name: MemoizationPolicy
# - name: RulePolicy
# - name: UnexpecTEDIntentPolicy
# max_history: 5
# epochs: 100
# - name: TEDPolicy
# max_history: 5
# epochs: 100
# constrain_similarities: true
The configuration parameters for your Rasa project are contained in this file. Here is a breakdown of the configuration file:
1. Assistant ID:
Assistant_id: placeholder_default
This placeholder value should be replaced with a unique identifier for your assistant.
2. Rasa NLU configuration:
Language: en
Specifies the language used for natural language understanding.
Pipeline:
Defines the pipeline of components used for NLU processing.
The pipeline is currently commented out, and the default pipeline is used.
The default pipeline includes various components like tokenizers, featurizers, classifiers, and response selectors.
If you want to customize the pipeline, you can uncomment the lines and adjust the pipeline configuration.
3. Rasa core configuration:
Policies:
Specifies the policies used for dialogue management.
The policies are currently commented out, and the default policies are used.
The default policies include memoization, rule-based, and TED (Transformer Embedding Dialogue) policies
If you want to customize the policies, you can uncomment the lines and adjust the policy configuration.
The custom actions that your chatbot can execute are contained in this file. Retrieving data from an API, communicating with a database, or doing any other unique business logic are all examples of actions.
# This files contains your custom actions which can be used to run
# custom Python code.
#
# See this guide on how to implement these action:
# https://rasa.com/docs/rasa/custom-actions
# This is a simple example for a custom action which utters "Hello World!"
# from typing import Any, Text, Dict, List
#
# from rasa_sdk import Action, Tracker
# from rasa_sdk.executor import CollectingDispatcher
#
#
# class ActionHelloWorld(Action):
#
# def name(self) -> Text:
# return "action_hello_world"
#
# def run(self, dispatcher: CollectingDispatcher,
# tracker: Tracker,
# domain: Dict[Text, Any]) -> List[Dict[Text, Any]]:
#
# dispatcher.utter_message(text="Hello World!")
#
# return []
Explanation of the code:
The ActionHelloWorld class extends the Action class provided by the rasa_sdk.
The name method defines the name of the custom action, which in this case is “action_hello_world”.
The run method is where the logic for the custom action is implemented.
Within the run method, the dispatcher object is used to send a message back to the user. In this example, the message sent is “Hello World!”.
The return [] statement indicates that the custom action has completed its execution.
The endpoints for your chatbot are specified in this file, including any external services or the webhook URL for any custom actions.
Initializing Rasa
This command initializes a new Rasa project in the current directory ($(pwd)):
docker run -p 5005:5005 -v $(pwd):/app rasa/rasa:3.5.2 init --no-prompt
It sets up the basic directory structure and creates essential files for a Rasa project, such as config.yml, domain.yml, and data/nlu.yml. The -p flag maps port 5005 inside the container to the same port on the host, allowing you to access the Rasa server. <IMAGE>:3.5.2 refers to the Docker image for the specific version of Rasa you want to use.
Training the model
The following command trains a Rasa model using the data and configuration specified in the project directory:
docker run -v $(pwd):/app rasa/rasa:3.5.2 train --domain domain.yml --data data --out models
The -v flag mounts the current directory ($(pwd)) inside the container, allowing access to the project files. The --domain domain.yml flag specifies the domain configuration file, --data data points to the directory containing the training data, and --out models specifies the output directory where the trained model will be saved.
Running the model
This command runs the trained Rasa model in interactive mode, enabling you to test the chatbot’s responses:
docker run -v $(pwd):/app rasa/rasa:3.5.2 shell
The command loads the trained model from the models directory in the current project directory ($(pwd)). The chatbot will be accessible in the terminal, allowing you to have interactive conversations and see the model’s responses.
Verify Rasa is running:
curl localhost:5005
Hello from Rasa: 3.5.2
Now you can send the message and test your model with curl:
curl --location 'http://localhost:5005/webhooks/rest/webhook' \
--header 'Content-Type: application/json' \
--data '{
"sender": "Test person",
"message": "how are you ?"}'
Running WebChat
The following command deploys the trained Rasa model as a server accessible via a WebChat UI:
docker run -p 5005:5005 -v $(pwd):/app rasa/rasa:3.5.2 run -m models --enable-api --cors "*" --debug
The -p flag maps port 5005 inside the container to the same port on the host, making the Rasa server accessible. The -m models flag specifies the directory containing the trained model. The --enable-api flag enables the Rasa API, allowing external applications to interact with the chatbot. The --cors "*" flag enables cross-origin resource sharing (CORS) to handle requests from different domains. The --debug flag enables debug mode for enhanced logging and troubleshooting.
docker run -p 8080:80 harshmanvar/docker-ml-faq-rasa:webchat
The rasa service is based on the rasa/rasa:3.5.2 image.
It exposes port 5005 to communicate with the Rasa API.
The current directory (./) is mounted as a volume inside the container, allowing the Rasa project files to be accessible.
The command run -m models --enable-api --cors "*" --debug starts the Rasa server with the specified options.
The webchat service is based on the harshmanvar/docker-ml-faq-rasa:webchat image. It builds the image using the Dockerfile-webchat file located in the current context (.). Port 8080 on the host is mapped to port 80 inside the container to access the webchat interface.
You can clone the repository or download the docker-compose.yml file directly from GitHub.
Bringing up the container services
You can start the WebChat application by running the following command:
docker compose up -d --build
Then, use the docker compose ps command to confirm that your stack is running properly. Your terminal will produce the following output:
docker compose ps
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
docker-ml-faq-rassa-rasa-1 harshmanvar/docker-ml-faq-rasa:3.5.2 "rasa run -m models …" rasa 6 seconds ago Up 5 seconds 0.0.0.0:5005->5005/tcp
docker-ml-faq-rassa-webchat-1 docker-ml-faq-rassa-webchat "/docker-entrypoint.…" webchat 6 seconds ago Up 5 seconds 0.0.0.0:8080->80/tcp
Viewing the containers via Docker Dashboard
You can also leverage the Docker Dashboard to view your container’s ID and easily access or manage your application (Figure 4):
Figure 4: View containers with Docker Dashboard.
Conclusion
Congratulations! You’ve learned how to containerize a Rasa application with Docker. With a single YAML file, we’ve demonstrated how Docker Compose helps you quickly build and deploy an ML FAQ Demo Model app in seconds. With just a few extra steps, you can apply this tutorial while building applications with even greater complexity. Happy developing.
Observability, especially in Kubernetes, often results in a bunch of different tools. One for metrics, another for events, something for tracing, something else for logging, and so on. Pixie tries to change that with an all-in-one solution based on eBPF and scripting.
▬▬▬▬▬▬ Sponsoships ▬▬▬▬▬▬ If you are interested in sponsoring this channel, please use https://calendly.com/vfarcic/meet to book a timeslot that suits and we’ll go over the details. Or feel free to contact me over Twitter or LinkedIn (see below)
Connexion
