git and CI/CD workflow¶
Continual is an operational AI platform and is designed to be GitOps-friendly. Users who routinely work in Git should feel comfortable leveraging Continual's commands to fit into their continuous integration and continuous deployment development and production workflows. Before getting started here, we recommend reading our section on Environments, which provide essential "branching" functionality that enables many of these workflows.
Using environments for dev and prod¶
When users wish to iterate on a project in Continual, there are a few things that are required:
Create a separate environment, isolated from production.
Copy all resources from production into the new environment.
git, this would be embodied by the command
git checkout -b <my_branch>.
In continual, you can similarly execute the following command:
continual checkout <my_env>
This command create a new environment and syncs that environment with
production. Users can then iterate on the new environment without affecting
continual. When using checkout, Continual creates a new schema in your feature
<project_id>_<env_id>. Which will contain any resources created in the
new environment. This ensures that your schemas are always distinct from those
used and production and never clash.
If you wish to build an environment without syncing production over, you can do so with the normal
continual environments create command.
Building an environment¶
Now that our environment is created, we'll want to be able to push our changes
into the environment. To accomplish this, we'll just use the normal
continual push command:
continual push <my_dir>
This pushes the specified files into Continual and any modified files will
trigger a build. By deafult this uses the current project and environment, but
you can manually specify those as well via
--project <project_id> --env <env_id>.
You'll be able to view any new resources in the Web UI or CLI as usual, in the
Note, you can use
--train-all to force behavior when
pushing content into Continual. This is particularly useful when working in a
branch if you want to force update models, even if their definitions have not
changed, or if you want to prevent re-trainings while doing an update.
As resources are modified and pushed into Continual, each is tracked in a
change object. You can think of these as akin to commits. Changes are
separated by environment and you can always view changes via:
continual changes list --project <project_id> --env <env_id>
or, by visiting the
Changes section of the Web UI.
Any failed change can be rebuilt, either in the Web UI or via
continual changes rerun <change_id>.
Merging an environment¶
The last tool we need at our disposal is some way to merge an environment back
into production. In Git, we have
git merge, and Continual also provides the
continual merge --from <env_id>
This command now merges the changes in your environment into the production
environment. Any changed or new feature-sets or models will be built. By
merge executes in the production environment. If you wish to merge to
a non-prod environment, you can do so via
--to <env_id>. Note that you cannot
merge across projects.
You can use
merge to see what changes will be maid
during a merge without actually executing the merge.
Via the Environments section in the Web UI, you may also merge an environment into another:
Using Continual with your CI/CD¶
With the above commands, it should be relatively straight forward to integrate Continual into your CI/CD pipelines. For those wishing to do so, we offer the following recommendations:
Use a separate Git repository for each Continual project. Name the repository to be the same as the project ID.
If an environment is not specified when using the Continual CLI, Continual will try to use your git branch as the environment name. This is a safety mechanism to prevent unintended modifications to produtction. For Git users, this should minimize the amount of options you need to enter for various commands. If you wish to explicitly use an environment different than your branch, you can either pass in
--envor set the default environment via
continual config set-environment.
You should configure your CI/CD pipeline to build off of pull requests and execute
continual push <continual-dir>into the
repo@branchenvironment in Continual. Remember that the repo corresponds to your project_id and the branch will correspond to the environment name. Subsequent commits into the PR will push into the same environment and will only build modified resources.
Configure your CI/CD pipeline so that when a PR is approved,
continual merge -from repo@branchis executed. This will trigger Continual to push all modifications to the production environment.
git clone <my-dbt-project> cd <my-dbt-project> git checkout -b <my-new-branch> continual checkout <my-new-branch> <modify Continual files> continual push --env <my-new-branch> <modified-files> git add <modified dbt files> git commit -a git push
Production no CI/CD¶
git clone <my-dbt-project> cd <my-dbt-project> git checkout -b <my-new-branch> continual checkout <my-new-branch> <modify Continual files> continual push --env <my-new-branch> <modified-files> git add <modified dbt files> git commit -a git push continual push #pushes into prod environment
Production with CI/CD¶
git clone <my-dbt-project> cd <my-dbt-project> git checkout -b <my-new-branch> continual checkout <my-new-branch> <modify Continual files> git add <modified dbt files> git commit -a git push <create pull request> <CI/CD executes `continual push` on branch env> <review + approve pr> <CI/CD executes `continual merge` into production>