API Documentation for nua-orchestrator¶
Package nua.orchestrator¶
Module nua.orchestrator.__main__¶
Module nua.orchestrator.admin_local¶
generate_rsa_host_key ¶
generate_rsa_host_key(bits=4096)
Generate a new private RSA key.
This factory function can be used to generate a new host key or authentication key.
private_key_blob_from_key ¶
private_key_blob_from_key(key, password=None)
Write an SSH2-format private key file in a form that can be read by paramiko or
openssh.
If no password is given, the key is written in a trivially-encoded format (base64)
which is completely insecure. If a password is given, DES-EDE3-CBC is used.
set_new_host_key ¶
set_new_host_key()
Generate a random RSA key (4096 bits) and store it in the nua DB settings as
“host.host_priv_key_blob”.
set_new_host_key_from_file ¶
set_new_host_key_from_file()
Read private RSA ey from file and store it in the nua DB settings as
“host.host_priv_key_blob”.
Module nua.orchestrator.api¶
API to access orchestrator commands.
API ¶
API()
API to access orchestrator commands.
Module nua.orchestrator.app_deployer¶
Class to manage the deployment of a group of app instances.
Note
- Security of deployment (deploy an app instance upon an existing one) is not
currently managed in this level.
AppDeployer ¶
AppDeployer()
Deployment of a list of app instance/nua-image.
Devel notes: will be refactored into base class and subclasses for various
deployment strategies (restoring previous configuration, …).
example of use
deployer = AppDeployment()
deployer.local_services_inventory()
deployer.load_deploy_config(deploy_config)
deployer.gather_requirements()
deployer.configure_apps()
deployer.deactivate_previous_apps()
deployer.apply_nginx_configuration()
deployer.start_apps()
deployer.post_deployment()
local_services_inventory ¶
local_services_inventory()
Initialization step: inventory of available providers available on
the host, like local databases.
merge ¶
merge(additional, option='add')
Merge a deployment configuration into the current deployed configuration.
WIP
- first step: ‘add’, strict, no conflict allowed
- next: add and replace if needed (merge)
merge_sequential ¶
merge_sequential(additional)
Merge a deployment configuration sequentially.
Difference from merge(): all (usually one) additional apps are added one nby
one.
evaluate_deploy_strategy ¶
evaluate_deploy_strategy(merged_app)
Return the name of the method to use to deploy the “new” app.
Expected result if condition
label known:
“deploy_update_app” : same app on same label and domain
“deploy_replace_app” : another app on same label and domain
“deploy_move_domain” : same app on same label and a unused domain
raise Abort : same app on same label but another already used domain
“deploy_reuse_label” : another app on same label but another domain
new label:
raise Abort : if the domain is a domain already in use
“deploy_new_app” : new label and unused domain
configure_apps_step2 ¶
configure_apps_step2()
“Second part of app configuration: common data (ports).
read_initial_volumes ¶
read_initial_volumes()
Read the initial volumes to display that information after operations.
restore_deactivate_previous_apps ¶
restore_deactivate_previous_apps()
For restore situation, find all instance in DB.
- remove container if exists
- remove site from Nua DB
apply_nginx_configuration ¶
apply_nginx_configuration()
Apply configuration to Nginx, especially configurations that can not be
deployed before all previous apps are stopped.
merge_nginx_configuration ¶
merge_nginx_configuration()
Apply configuration to Nginx, for new apps and existing ones.
remove_nginx_configuration ¶
remove_nginx_configuration(stop_domain)
Remove apps from the nginx configuration.
To stop nginx redirection before actually stopping the apps.
remove_all_deployed_nginx_configuration ¶
remove_all_deployed_nginx_configuration()
Remove all deployed apps from the nginx configuration.
To stop nginx redirection before actually stopping the apps.
stop_all_deployed_apps ¶
stop_all_deployed_apps(store_status=False)
Stop all deployed app instances.
remove_container_and_network ¶
remove_container_and_network(apps)
Remove stopped app: container, network, but not volumes.
remove_all_deployed_container_and_network ¶
remove_all_deployed_container_and_network()
Remove all (stopped) apps container, network, but not volumes.
remove_managed_volumes ¶
remove_managed_volumes(apps)
Remove data of stopped app: local managed volumes.
remove_all_deployed_managed_volumes ¶
remove_all_deployed_managed_volumes()
Remove local volumes of all (stopped) apps.
remove_deployed_instance ¶
remove_deployed_instance(apps)
Remove data of stopped app: local managed volumes.
parse_deploy_apps ¶
parse_deploy_apps()
Make the list of AppInstances to be deployed/merged.
Check config syntax, replace missing information by defaults.
sort_apps_per_name_domain ¶
sort_apps_per_name_domain()
Classify the apps per domain, filtering out miss declared apps.
The apps per domain are available in self.apps_per_domain
apps_merge_app_instances_to_providers ¶
apps_merge_app_instances_to_providers()
Merge configuration declared in the AppInstance config to original nua-config
declarations.
configured_ports ¶
configured_ports()
Return set of required host ports (aka non auto ports) from site_list.
Returns: set of integers
evaluate_container_params ¶
evaluate_container_params(app)
Compute site run environment parameters except those requiring late
evaluation (i.e. host names of started containers).
generate_app_container_run_parameters ¶
generate_app_container_run_parameters(app)
Return suitable parameters for the docker.run() command.
Does not include the internal_secrets, that are passed only at docker.run()
execution, thus secrets not stored in instance data.
generate_provider_container_run_parameters ¶
generate_provider_container_run_parameters(provider)
Return suitable parameters for the docker.run() command (for Provider).
sanitize_run_params
staticmethod
¶
sanitize_run_params(run_params)
Docker constraint: 2 docker options not compatible.
display_container_status ¶
display_container_status(app)
Display current state of the caontainer (running, exited…).
Module nua.orchestrator.app_instance¶
AppInstance ¶
AppInstance(app_instance_dict)
providers
property
writable
¶
providers
List of sub providers of the AppInstance object.
Warning: only AppInstance class has an actual use of ‘providers’.
The subclass Provider will always provide an empty list
image_short
property
¶
image_short
Return short app id from deployment ‘image’ value.
Used early: at that moment the Docker image and actual
image/nua_config are not available.
Remove ‘nua-’ prefix and version): “nua-hedgedoc” ->
“hedgedoc”.
nua_tag
property
¶
nua_tag
Return long tag string with version and release.
“hedgedoc” -> “nua-hedgedoc:1.9.7-3”
backup_records_crop ¶
backup_records_crop(max_length=7, max_age=31)
Kepp in the backup_records list the last max_length elements and remove the
elements olders than max_age.
Note: clean also the local backup directory?
persistent ¶
persistent(name)
Return Persistent instance for provider of name ‘name’.
Use name = ‘’ for main site.
set_network_name ¶
set_network_name()
Evaluate the need of a bridge private network.
If needed, set a relevant network name.
rebase_env_upon_nua_conf ¶
rebase_env_upon_nua_conf()
Merge AppInstance declared env at deploy time upon base declaration of nua-
config.
order_providers_dependencies ¶
order_providers_dependencies()
Order of evaluations for variables.
- main AppInstance variable assignment (including hostname of providers)
- late evaluation (hostnames)
And check for circular dependencies of providers.
parse_providers ¶
parse_providers()
Build the providers list or Provider from nua_config.
The providers will be later updated from site config providers statements.
Note: there is still a “provider” key for instance changes.
default_label ¶
default_label()
Return a label based on app id and domain.
To use when the user does not provide an app label or as default value.
set_providers_names ¶
set_providers_names()
Set first container names of providers to permit early host assignment to
variables.
Module nua.orchestrator.app_manager¶
Class to manage the currently active apps.
General use
- load active configuration
- loop over apps to perform an action (example: backup)
AppManager ¶
AppManager()
Management of a list of site.
backup_all_apps ¶
backup_all_apps()
Execute a one-time backup for all site instance having a backup
declaration.
backup_app_domain ¶
backup_app_domain(domain)
Execute a one-time backup for the apps instance of domain.
backup_one_app ¶
backup_one_app(app)
Execute a full backup of an app (all volumes).
Backup order:
1 - providers of app
2 - app
for each:
a) backup tag of each volume
b) backup tag of main provider
restore_backup_app_per_label ¶
restore_backup_app_per_label(label)
Execute a backup restoration.
It is assumed that the app is stopped.
restore_backup_app_per_domain ¶
restore_backup_app_per_domain(domain)
Execute a backup restoration.
It is assumed that the app is stopped.
restore_list_backups_app_per_label ¶
restore_list_backups_app_per_label(label)
List available backups for the app.
restore_list_backups_app_per_domain ¶
restore_list_backups_app_per_domain(domain)
List available backups for the app.
Package nua.orchestrator.assign¶
Module nua.orchestrator.assign.engine¶
Functions to respond to requirement from instance declarations.
instance_key_evaluator ¶
instance_key_evaluator(app, provider=None, late_evaluation=False, port=None)
Evaluate value for ‘env’ values declared as dict with dynamic parameters, through
retrieving persistent value or compute value from specialized functions.
Module nua.orchestrator.assign.evaluators¶
Functions to respond to requirement from instance declarations.
All evaluation function must have same 2 arguments:
function(provider, requirement) but call through wrapper uses a third
argument: ‘persistent’
persistent_value ¶
persistent_value(func)
Store automatic generated values for next deployment of the same image.
Default is ‘persistent = true’. If persistent is False, erase the data from local
config storage.
random ¶
random(provider, destination_key, requirement)
Send a random string or a random integer.
The value is either ramdomly generated or read from previous execution if
‘persistent’ is true (default) and previous data is found. Default length for random
string is 24. Random integer is a 64 bit positive signed, [0, 2*64-1]
unique_user ¶
unique_user(provider, destination_key, requirement)
Send a unique user id (for DB creation).
- sequential generated,
- or from previous execution if ‘persistent’ is true and previous
data is found.
unique_db ¶
unique_db(provider, destination_key, requirement)
Send a unique DB id (for DB creation).
- sequential generated,
- or from previous execution if ‘persistent’ is true and previous
data is found.
provider_property ¶
provider_property(rsite, destination_key, requirement)
Retrieve value from provider by name.
Example
CMD_DB_HOST = { from=”“, key=”hostname” }
CMD_DB_HOST = { from=”database”, key=”hostname” }
CMD_DB_DATABASE = { from=”database”, key=”POSTGRES_DB” }
nua_internal ¶
nua_internal(rsite, destination_key, requirement)
Retrieve key from nua_internal values, do not store the value in instance
configuration.
The value is only set when executing the docker.run() for main site and all sub
providers.
external_ip_evaluation ¶
external_ip_evaluation(_unused, destination_key, requirement)
Return the detected external IP address (v4).
The value is only set when executing the docker.run() for main site and all sub
providers.
Module nua.orchestrator.assign.gen_db_credentials¶
Package nua.orchestrator.backup¶
Module nua.orchestrator.backup.app_backup¶
AppBackup
dataclass
¶
Class to control the backup of an app and its volumes.
Module nua.orchestrator.backup.app_restore¶
AppRestore
dataclass
¶
Class to control the restoration from backup of an app instance.
Module nua.orchestrator.backup.backup_component¶
BackupComponent
dataclass
¶
Single backup up unit (typically a directory or a database) and its restore
method.
One item has
- a path (or url)
- a restore method (name of the backup plugin)
- a backup date
Module nua.orchestrator.backup.backup_record¶
BackupRecord
dataclass
¶
Record of a successful backup of the data of an instance.
One record
- label_id of the app
- unique reference date (end date, iso format)
- list of backuped components and their restore method
Module nua.orchestrator.backup.backup_registry¶
Module nua.orchestrator.backup.backup_report¶
Package nua.orchestrator.backup.plugins¶
Module nua.orchestrator.backup.plugins.mongodump¶
Class to backup a MongoDB database.
BckMongodump ¶
BckMongodump(provider, volume=None, ref_date='')
Module nua.orchestrator.backup.plugins.mysqldump¶
Class to backup a Mariadb database.
BckMysqldump ¶
BckMysqldump(provider, volume=None, ref_date='')
Module nua.orchestrator.backup.plugins.pg_dump¶
Class to backup a database.
BckPostgresDump ¶
BckPostgresDump(provider, volume=None, ref_date='')
Module nua.orchestrator.backup.plugins.plugin_base_class¶
Module nua.orchestrator.backup.plugins.tgz_volumes¶
Class to backup the volumes of a container.
BckTgzVolumes ¶
BckTgzVolumes(provider, volume=None, ref_date='')
Module nua.orchestrator.backup.provider_backup¶
backup_provider ¶
backup_provider(provider, ref_date='')
Execute a backup from main ‘backup’ configuration of a Provider.
Module nua.orchestrator.backup.volume_backup¶
backup_volume ¶
backup_volume(provider, volume, ref_date='')
Execute a backup from backup tag of a Volume of a Provider.
Package nua.orchestrator.bootstrap¶
Module nua.orchestrator.bootstrap.bootstrap¶
Bootstrap Nua orchestrator on the local host.
Use this script for installing the orchestrator on a new host.
- Create Nua account with admin rights,
- install base packages and configuration.
In future versions, change this to a standalone script.
Package nua.orchestrator.certbot¶
Module nua.orchestrator.certbot.certbot¶
Certbot main entry, detect strategy and apply.
host.certbot_strategy (or ENV NUA_CERTBOT_STRATEGY) can be (for now):
- auto: all domains are declared to certbot to use HTTPS
- none: no HTTPS domain, all is converted to HTTP only (i.e. tests or local server)
- Default is “auto”
Test ENV variables
- NUA_CERTBOT_VERBOSE: show certbot log
- NUA_CERTBOT_TEST: use certbot test environment (only for tests)
register_certbot_domains_per_domain ¶
register_certbot_domains_per_domain(apps_per_domain)
Apply certbot strategy to domains, filtering out internal deployments.
input format:
[{‘hostname’: ‘test.example.com’,
‘apps’: [{‘domain’: ‘test.example.com/instance1’,
‘image’: ‘flask-one:1.2-1’,
},
{‘domain’: ‘test.example.com/instance2’,
‘image’: ‘flask-one:1.2-1’,
},
…
register_certbot_domains ¶
register_certbot_domains(apps)
Apply certbot strategy to domains.
Group common domains and execute “certbot run”.
(Only public function).
Warning:
- Top domain “exemple.com” is NOT configurd for cerbot if only
“www.exemple.com” is listed,
- all sub-domains “xxx.exemple.com” share the same key.
Module nua.orchestrator.certbot.commands¶
Certbot strategies.
host.certbot_strategy (or ENV NUA_CERTBOT_STRATEGY) can be (for now):
- auto: all domains are declared to certbot to use HTTPS
- none: no HTTPS domain, all is converted to HTTP only (i.e. tests or local server)
- Default is "auto"
Test ENV variables:
- NUA_CERTBOT_VERBOSE: show certbot log
- NUA_CERTBOT_TEST: use certbot test environment (only for tests)
certbot_certonly_command ¶
certbot_certonly_command(domain, option)
Build cerbot’s arguments for a subdomains.
Standalone or nginx call.
apply_auto_strategy ¶
apply_auto_strategy(top_domain, domains)
Convert just created HTTP configuration (by nginx template) to HTTPS using
certbot (if strategy is ‘auto’).
Each domain of the list uses the same SSL key.
- implementation mode “auto”:
- let certbot rewrite redirections,
- let certbot manage cron,
- apply “auto” rules and parameters.
Package nua.orchestrator.certbot.config¶
Module nua.orchestrator.certbot.installer¶
Certbot package installer.
Install Certbot config files for the nua user at bootstrap time.
TODO: add a cron to revoke domains not instancied sinc a few days.
Package nua.orchestrator.cli¶
Package nua.orchestrator.cli.commands¶
Module nua.orchestrator.cli.commands.backup_restore¶
Execute backup commands.
backup_all_apps ¶
backup_all_apps()
Execute a one-time backup for all instances having a backup declaration.
backup_one_app ¶
backup_one_app(*, label='', domain='')
Execute a one-time backup for the app instance identified by its label.
restore_last_backup ¶
restore_last_backup(*, label='', domain='')
Restore last backuped data for the app instance identified by its label or
domain.
restore_list_backups ¶
restore_list_backups(*, label='', domain='')
List available backups for the app instance identified by its label or domain.
Module nua.orchestrator.cli.commands.deploy_remove¶
Nua main scripts.
remove_nua_domain ¶
remove_nua_domain(domain, state_journal)
Remove some deployed app instance, erasing its data and container.
Deprecated: requires identification of the instance per domain name.
remove_nua_label ¶
remove_nua_label(label, state_journal)
Remove some deployed app instance, erasing its data and container.
deploy_merge_nua_app ¶
deploy_merge_nua_app(merge_config, state_journal)
Add somme app config to the deployed list.
deploy_merge_one_nua_app_config ¶
deploy_merge_one_nua_app_config(app_config, state_journal)
Add somme app config to the deployed list.
Module nua.orchestrator.cli.commands.restore_deployed¶
Restore previous successful deployed configuration.
restore_active_state ¶
restore_active_state()
Restore to the most recent deployment configuration that did succeed.
Module nua.orchestrator.cli.commands.start_stop¶
Start/stop/restart some deployed app instance.
stop_nua_instance ¶
stop_nua_instance(*, label='', domain='')
Stop some deployed app instance per label or per domain.
start_nua_instance ¶
start_nua_instance(*, label='', domain='')
Start some deployed app instance per label or per domain.
restart_nua_instance ¶
restart_nua_instance(*, label='', domain='')
Restart some deployed app instance per label or per domain.
start_nua_instance_domain ¶
start_nua_instance_domain(domain, state_journal)
Start some deployed app instance (per domain).
The instance is started (if it was already deployed).
start_nua_instance_label ¶
start_nua_instance_label(label, state_journal)
Start some deployed app instance (per label).
The instance is started (if it was already deployed).
stop_nua_instance_domain ¶
stop_nua_instance_domain(domain, state_journal)
Stop some deployed app instance (per domain).
The instance is stopped, but not uninstalled (volumes are kept).
stop_nua_instance_label ¶
stop_nua_instance_label(label, state_journal)
Stop some deployed app instance (per label).
The instance is stopped, but not uninstalled (volumes are kept).
restart_nua_instance_domain ¶
restart_nua_instance_domain(domain, state_journal)
Restart some deployed app instance (per domain).
restart_nua_instance_label ¶
restart_nua_instance_label(label, state_journal)
Restart some deployed app instance (per label).
pause_nua_instance ¶
pause_nua_instance(*, label='', domain='')
Stop some deployed app instance per label or per domain.
unpause_nua_instance ¶
unpause_nua_instance(*, label='', domain='')
Start some deployed app instance per label or per domain.
pause_nua_instance_domain ¶
pause_nua_instance_domain(domain, state_journal)
Pause some deployed app instance (per domain).
pause_nua_instance_label ¶
pause_nua_instance_label(label, state_journal)
Pause some deployed app instance (per label).
unpause_nua_instance_domain ¶
unpause_nua_instance_domain(domain, state_journal)
Unpause some deployed app instance (per domain).
unpause_nua_instance_label ¶
unpause_nua_instance_label(label, state_journal)
Unpause some deployed app instance (per label).
Module nua.orchestrator.cli.commands.status¶
Module nua.orchestrator.cli.configuration¶
Module nua.orchestrator.cli.debug¶
Module nua.orchestrator.cli.main¶
Script main entry point for Nua local.
deploy_local ¶
deploy_local(apps_conf=arg_deploy_app, verbose=opt_verbose, colorize=option_color)
Deploy one or several Nua applications.
deploy_replace_local ¶
deploy_replace_local(
apps_conf=arg_deploy_app, verbose=opt_verbose, colorize=option_color
)
Replace all deployed instances by new deployment list.
remove_local ¶
remove_local(
verbose=opt_verbose, colorize=option_color, label=option_label, domain=option_domain
)
Remove a deployed instance and all its data.
restore_local ¶
restore_local(verbose=opt_verbose, colorize=option_color)
Restore last successful deployment.
stop_local ¶
stop_local(
verbose=opt_verbose, colorize=option_color, label=option_label, domain=option_domain
)
Stop a deployed instance.
start_local ¶
start_local(
verbose=opt_verbose, colorize=option_color, label=option_label, domain=option_domain
)
Start a deployed instance.
restart_local ¶
restart_local(
verbose=opt_verbose, colorize=option_color, label=option_label, domain=option_domain
)
Restart a deployed instance.
backup_cmd ¶
backup_cmd(
verbose=opt_verbose,
colorize=option_color,
all_apps=option_all_apps,
label=option_label,
domain=option_domain,
)
Backup app instance(s) having a backup rules.
restore_last_backup_cmd ¶
restore_last_backup_cmd(
verbose=opt_verbose,
colorize=option_color,
label=option_label,
domain=option_domain,
list_flag=option_list_backup,
last_flag=option_last_backup,
)
Restore backuped data for the app instance.
Module nua.orchestrator.config¶
Configuration of the nua orchestrator server.
‘config’ is an DeepAccessDict(), if a dict is needed, use config.read()
Module nua.orchestrator.constants¶
Package nua.orchestrator.db¶
Module nua.orchestrator.db.create¶
Package nua.orchestrator.db.model¶
Module nua.orchestrator.db.model.auth¶
Module nua.orchestrator.db.model.base¶
Module nua.orchestrator.db.model.deployconfig¶
DeployConfig ¶
The deployment configuration.
The configuration requested from the last “nua deploy” commands, with timestamp
and status.
- id: system generated colum for indexes
- previous: id of previous deployment (or zero)
- state: one of “failed”, “active”, “inactive”, “previous”. “previous” is an
inactive state for last running config. - created, modified: status date
- deploy_config: JSON data representation of deployment config, ex:
{
“requested”: {“site”:[ # original .toml request
{
“image”: “hedgedoc:1.9.6-4”,
“domain”: “test1.example.com”,
“env”: {
“CMD_DOMAIN”: “test1.example.com”
}
},
{
“image”: “hedgedoc:1.9.6-4”,
“domain”: “test2.example.com”,
“env”: {
“CMD_DOMAIN”: “test2.example.com”
}
}
]
},
“deployed”: [
{ # detailed configuration, ‘self.apps’ content
“image”: “hedgedoc:1.9.6-4”,
“domain”: “test1.example.com”,
…
}
]
}
Module nua.orchestrator.db.model.host¶
Host ¶
Contains the Nua host information.
Note: Could be also stored in the Na app blob. Thus, not clear if that
table will be used.
Module nua.orchestrator.db.model.image¶
Image ¶
A Docker image managed by Nua.
Map of some docker image parameters and Nua tags.
id_sha: docker id of image, “sha256:abc123…”
nua_tag: full Nua tag: “nua-apache-basic:2.4.52-2”
app_id: id of the Nua app: “apache-basic”
nua_version: version of Nua that created the image: “0.1.38”
created: date creation, iso format, local: “2022-06-14T15:22:11”
size: size in bytes
FIXME: created to be improved with tz info)
Module nua.orchestrator.db.model.instance¶
Instance ¶
The deployed instance of an app.
- id: system generated colum for indexes
- app_id: nua id of the app
- label_id: label of the instance
- nua_tag: nua tag, “nua-{app_id}:{version}-{release}”. nua_tag permits
to find the related docker image. - domain: domain serving the app
- container: name of the deployed contaner if image started (or “”)
- image: name of the docker image (if docker deployed)
- state: one of “running” “stopped”
- site_config: JSON data representation of actual deployment config
values, including the instance nginx domain.
{‘host_use’: 8109,
‘container’: ‘nua-flask-upload-one-1.0-1-sloop.example.com’,
‘domain’: ‘sloop.example.com/xxx’,
‘image’: ‘nua-flask-upload-one:1.0-1’,
‘image_id’: ‘sha256:232d921796c7f62f9240d8727d39829d31772a395d5c060ece5c74a6315b2f0e’,
‘image_nua_config’: {‘build’: {‘document_root’: ‘/nua/app/html’},
‘instance’: {‘port’: 5100, domain…},
‘metadata’: {‘author’: ‘Nua testers’,
‘id’: ‘flask-upload-one’,
‘licence’: ‘MIT’,
‘profile’: [‘test’],
‘release’: 1,
‘tagline’: ‘Nua test with Flask and mount ‘
‘- one’,
‘tags’: [‘test’,
‘html’,
‘web’,
‘static’,
‘mount’],
‘title’: ‘Flask test upload one’,
‘version’: ‘1.0’,
‘website’: ‘https://nua.rocks/’},
‘docker’: {‘auto_remove’: True,
‘detach’: True,
‘mem_limit’: ‘1G’,
‘mounts’: [{‘ReadOnly’: False,
‘Source’: ‘flask_uploads’,
‘Target’: ‘/var/tmp/uploads’,
‘Type’: ‘volume’}],
‘name’: ‘nua-flask-upload-one-1.0-1-sloop.example.com’,
‘ports’: {‘80/tcp’: 8109}},
‘env’: {},
‘volume’: [{‘driver’: ‘local’,
‘dst’: ‘/var/tmp/uploads’,
‘name’: ‘flask_uploads’,
‘type’: ‘volume’}]},
‘port’: ‘auto’,
‘run_params’: {‘auto_remove’: True,
‘detach’: True,
‘mem_limit’: ‘1G’,
‘mounts’: [{‘ReadOnly’: False,
‘Source’: ‘flask_uploads’,
‘Target’: ‘/var/tmp/uploads’,
‘Type’: ‘volume’}],
‘name’: ‘nua-flask-upload-one-1.0-1-sloop.example.com’,
‘ports’: {‘80/tcp’: 8109}}}
There can be only one app per domain.
Module nua.orchestrator.db.model.setting¶
Setting ¶
The settings of an instance of an app.
- id: system generated colum for indexes
- app_id: nua id of the app
- nua_tag: nua tag, “nua-{app_id}:{version}-{release}”. nua_tag permits
to find the related docker image. - instance: “” for the base settings of the app. “local” for
runnning/installed nua-build software. User choice for others. - activation: “docker” or “system” for the moment. How this app service
is installed
bad idea - active: bool, wether these settings are applied on a currently running¶
instance (either local or dockerised or other)
- container: for “docker” installation app, “” or the id of the active¶
container.
- data: ontains a JSON data representation of actual config values. For
raw “” instance, it is the original nua-config.toml content.
Module nua.orchestrator.db.model.user_count¶
UserCount ¶
Table to store the user count information.
Usage: increment counter to create a unique user id.
Module nua.orchestrator.db.session¶
Module nua.orchestrator.db.store¶
Requests to backend DB.
Requests are full transactions, and mask the actual DB implementation to the
application.
store_image ¶
store_image(
id_sha="",
app_id="",
nua_tag="",
created="",
size=0,
nua_version=nua_version,
instance="",
data=None,
)
Store a Nua image in the local DB (table ‘image’).
Also set the initial settings of the image in the ‘setting’ table.
installed_nua_settings ¶
installed_nua_settings()
Return the dictionnary of settings of the nua-orchestrator.
nua-orchestrator is not actually an app, but we use the settings facility to store
its configuration in the DB with the app_id ‘nua- orchestrator’ (NUA_ORCH_ID).
set_nua_settings ¶
set_nua_settings(setting_dict)
Set the dictionnary of settings of the nua-orchestrator.
nua-orchestrator is not actually an app, but we use the settings facility to store
its configuration in the DB with the app_id ‘nua- orchestrator’
installed_nua_version ¶
installed_nua_version()
Return the version of ‘nua-orchestrator’ stored in the DB settings.
store_instance ¶
store_instance(
app_id="",
label_id="",
nua_tag="",
domain="",
container="",
image="",
state=STOPPED,
site_config=None,
)
Store a Nua instance in the local DB (table ‘instance’).
list_instances_container_local_active_volumes ¶
list_instances_container_local_active_volumes()
Return list of local mounted volumes.
Volumes with properties:
- required by active instances,
- locally mounted (‘docker’ driver), ‘managed’ type)
- unique per ‘source’ key.
list_instances_container_active_volumes ¶
list_instances_container_active_volumes()
Return list of mounted volumes or mounted local directories.
Volumes with properties:
- required by active instances,
- unique per ‘full_name’ key.
ports_instances_domains ¶
ports_instances_domains()
Return dict(port:domain) configured in instance, wether the instance is running
or not.
instance_port ¶
instance_port(domain)
Return the (main?) instance port. Dubious.
remarq: currently this function is unused
instance_persistent ¶
instance_persistent(label_id)
Return the persistent dictionary if (or an empty dict if not found).
deploy_config_add_config ¶
deploy_config_add_config(deploy_config, previous_id, state)
Store a Nua deployment configuration in local DB (table ‘deployconfig’).
Return
deploy_config_last_status ¶
deploy_config_last_status(status, limit=2)
Retrieve the config with “active” state.
It should be only one.
deploy_config_active ¶
deploy_config_active()
Retrieve the config with “active” state.
It should be only one.
deploy_config_previous ¶
deploy_config_previous()
Retrieve the config with “previous” state.
It should be zero, or sometimes only one.
deploy_config_last_inactive ¶
deploy_config_last_inactive()
Retrieve the last config with “inactive” state.
Package nua.orchestrator.db_migration¶
Module nua.orchestrator.db_migration.migrations¶
Module nua.orchestrator.db_migration.tools¶
execute_cmd ¶
execute_cmd(cmd)
Execute some command on the DB.
Ex: ‘alter table table_name add column column_name VARCHAR(160)
Package nua.orchestrator.db_utils¶
Module nua.orchestrator.db_utils.mariadb_utils¶
Nua mariadb orchestrator commands.
WIP
To remove all mariadb packages::
sudo apt-get remove --purge mariadb-server-10.6 mariadb-client
sudo apt-get autoremove sudo apt-get autoclean
sudo rm /var/lib/mysql/ib_logfile0 sudo rm /var/lib/mysql/ib_logfile1
sudo apt-get install mariadb-server-10.6
sudo apt-get install libmariadb3 libmariadb-dev
mariadb_pwd ¶
mariadb_pwd()
Return the ‘root’ user DB password of mariadb.
- When used in container context, the env variable NUA_MARIADB_PASSWORD should
contain the password. - When used in nua-orchestrator context, read the password from local file.
For orchestrator context, assuming this function can only be used after password
was generated (or its another bug).
Rem.: No cache. Rarely used function and pwd can be changed.
set_mariadb_pwd ¶
set_mariadb_pwd(password, any_ip=True)
Set mariadb root password for local instance of mariadb.
The password is stored in clear in Nua home. In future version, it could be replaced
by SSL key, thus gaining the ability to have encyption of streams and expiration
date. Basically we need clear password somewhere. Since this password is only used
by Nua scripts (if Nua is the only user of local mariadb DB), it could also be
generated / erased at each invocation. No test of min password length in this
function.
bootstrap_install_mariadb ¶
bootstrap_install_mariadb()
Installing the required mariadb version locally.
Need to be executed as root at nua install stage.
allow_docker_connection ¶
allow_docker_connection()
Check at deploy time that the mariadb_hba.conf file permit connexion.
Must be run as root at bootstrap time.
Module nua.orchestrator.db_utils.postgres_utils¶
Nua postgresql related commands.
postgres_pwd ¶
postgres_pwd()
Return the ‘postgres’ user DB password.
- When used in container context, the env variable NUA_POSTGRES_PASSWORD should
contain the password. - When used in nua-orchestrator context, read the password from local file.
For orchestrator context, assuming this function can only be used after password
was generated (or its another bug).
Rem.: No cache. Rarely used function and password can be changed.
set_postgres_pwd ¶
set_postgres_pwd(password)
Set postgres password for local instance of postgres.
The password is stored in clear in Nua home. In future version, it could be replaced
by SSL key, thus gaining the ability to have encryption of streams and expiration
date. Basically we need clear password somewhere. Since this password is only used
by Nua scripts (if Nua is the only user of local postgres DB), it could also be
generated / erased at each invocation. Password could be stored in some file in the
postgres user home (a postgres feature). No test of min password length in this
function.
pg_run_environment ¶
pg_run_environment(_unused_site)
Return a dict of environ variable for docker.run().
Actually, returns the DB postges password. This function to be used in orchestrator
environment, thus the password will be read from host file.
bootstrap_install_postgres ¶
bootstrap_install_postgres()
Installing the requied Postgres version locally.
Need to be executed as root at nua install stage
pg_check_listening ¶
pg_check_listening(_unused_site=None)
Check at deploy time that the postgres daemon is listening on the gateway port of
the docker service (ip passed as parameter).
This is launched for every deployed image (so cached).
allow_docker_connection ¶
allow_docker_connection()
Check at deploy time that the pg_hba.conf file permit connexion.
Must be run as root at bootstrap time.
Package nua.orchestrator.default_conf¶
Module nua.orchestrator.deploy_utils¶
Class to manage the deployment of a group of AppInstance.
load_install_image ¶
load_install_image(image_path)
Install docker image (tar file) in local docker daemon.
Return: tuple(image_id, image_nua_config)
extra_host_gateway ¶
extra_host_gateway()
Sent an update for docker parameters ‘extra_hosts’:
host.docker.internal.
new_docker_driver_config ¶
new_docker_driver_config(volume)
Volume driver configuration for Docker.
Only valid for the ‘volume’ type.
deactivate_app ¶
deactivate_app(app)
Deactive containers of AppInstance and all sub Providers (updating orchestrator
DB).
deactivate_all_instances ¶
deactivate_all_instances()
Find all instances in DB.
- remove container if it exists
- remove site from DB
start_container_engine ¶
start_container_engine()
Ensure the containr system is running.
Currrently: only checking that Docker daemon is up.
create_container_private_network ¶
create_container_private_network(network_name)
Create a private network for the container (and it’s sub containers).
Currrently: only managing Docker bridge network.
remove_container_private_network ¶
remove_container_private_network(network_name)
Remove an existing private network.
Currrently: only managing Docker bridge network.
pull_provider_container ¶
pull_provider_container(provider)
Retrieve a provider container or get reference from cache.
Currrently: only managing Docker bridge network.
Module nua.orchestrator.docker_utils¶
Docker utils.
docker_container_of_name ¶
docker_container_of_name(name, client=None)
Return the Container of the given name or None if not found.
docker_container_status_record ¶
docker_container_status_record(container_id)
Return container status dict (per container Id).
docker_remove_container_previous ¶
docker_remove_container_previous(name, show_warning=True)
Remove container of full domain name from running container and DB.
docker_remove_prior_container_live ¶
docker_remove_prior_container_live(rsite)
Search & remove containers already configured for this same AppInstance or
Provider (running or stopped), from Docker.
Security feature: try to remove containers of exactly same name that
could be found in docker daemon:
docker_run ¶
docker_run(rsite, secrets)
Wrapper on top of the py-docker run() command.
Returns:
| Type | Description |
|---|---|
Container
|
The new started container. |
params_with_secrets_and_f_strings ¶
params_with_secrets_and_f_strings(params, secrets)
Complete the docker run() environment parameter with secrets.
docker_exec_stdout ¶
docker_exec_stdout(container, params, output)
Wrapper on top of the py-docker exec_run() command, capturing the output.
Defaults are:
cmd, stdout=True, stderr=True, stdin=False, tty=False, privileged=False,
user=’‘, detach=False, stream=False, socket=False, environment=None,
workdir=None, demux=False
Returns: None
docker_exec_no_output ¶
docker_exec_no_output(container, command)
Wrapper on top of the py-docker exec_run() command, not capturing the output.
Defaults are:
cmd, stdout=True, stderr=True, stdin=False, tty=False, privileged=False,
user=’‘, detach=False, stream=False, socket=False, environment=None,
workdir=None, demux=False
Returns: None
docker_exec_stdin ¶
docker_exec_stdin(container, cmd, input_file)
Wrapper on top of the py-docker exec_run() command, capturing file to stdin.
Defaults are:
cmd, stdout=True, stderr=True, stdin=False, tty=False, privileged=False,
user=’‘, detach=False, stream=False, socket=False, environment=None,
workdir=None, demux=False
docker_exec_checked ¶
docker_exec_checked(container, params, output)
Wrapper on top of the py-docker exec_run() command, capturing the output.
Write the binary output of run_exec to output buffered io, or raise Runtime Error.
Defaults are:
cmd, stdout=True, stderr=True, stdin=False, tty=False, privileged=False,
user=’‘, detach=False, stream=False, socket=False, environment=None,
workdir=None, demux=False
Returns: None
docker_volume_of_name ¶
docker_volume_of_name(name, client=None)
Return the DockerVolume of the given name or None if not found.
docker_volume_create_local_dir ¶
docker_volume_create_local_dir(volume)
For volumes of type “directory”, create a local directory on the host if needed.
May use more options.
docker_volume_create_or_use ¶
docker_volume_create_or_use(volume_params)
Return an useable/mountable docker volume.
The strategy depends on the volume type: “managed”, “directory”, or
“tmpfs”.
docker_volume_prune ¶
docker_volume_prune(volume_opt)
Remove a (previously mounted) local docker volume.
Beware: deleting data !
docker_network_remove_one ¶
docker_network_remove_one(network_name)
Prune a network identified by its name.
docker_network_by_name ¶
docker_network_by_name(network_name)
Return a network identified by its name.
install_plugin ¶
install_plugin(plugin_name)
Install Docker’s plugin (plugin for API of remote services).
Module nua.orchestrator.domain_split¶
Module nua.orchestrator.healthcheck¶
Module nua.orchestrator.init¶
Module nua.orchestrator.internal_secrets¶
Package nua.orchestrator.local_services¶
Module nua.orchestrator.local_services.local_service_base¶
Module nua.orchestrator.local_services.mariadb¶
Module nua.orchestrator.local_services.postgres¶
Module nua.orchestrator.local_services.service_loader¶
Package nua.orchestrator.net_utils¶
Module nua.orchestrator.net_utils.external_ip¶
Module nua.orchestrator.net_utils.ports¶
Package nua.orchestrator.nginx¶
Module nua.orchestrator.nginx.commands¶
Package nua.orchestrator.nginx.html¶
Package nua.orchestrator.nginx.html.css¶
Module nua.orchestrator.nginx.installer¶
Module nua.orchestrator.nginx.render_default¶
Nginx utils to install nginx config and adapt with app using nginx.
clean_nua_nginx_default_site ¶
clean_nua_nginx_default_site()
Remove previous nginx sites.
Warning: only for user ‘nua’ or ‘root’
Module nua.orchestrator.nginx.render_site¶
Nginx utils to install nginx config and adapt with app using nginx.
remove_nginx_host_configuration ¶
remove_nginx_host_configuration(stop_domain)
Remove configuration for dommain.
warning: only for user ‘nua’ or ‘root’
configure_nginx_host ¶
configure_nginx_host(host)
Configure Nginx for the host passed as argument.
warning: only for user ‘nua’ or ‘root’
host format
{‘hostname’: ‘test.example.com’,
‘located’: True,
‘apps’: [{‘domain’: ‘test.example.com/instance1’,
‘hostname’: ‘example.com’,
‘top_domain’: ‘example.com’,
‘image’: ‘flask-one:1.2-1’,
‘location’: ‘instance1’
‘port’: {},
‘port_list’: [{‘container’: 3000,
‘host’: None,
‘host_use’: 8101,
‘name’: ‘web’,
‘protocol’: ‘tcp’,
‘proxy’: None,
‘web’: True,
‘ssl’: True}],
...
Package nua.orchestrator.nginx.templates¶
Module nua.orchestrator.nua_db_setup¶
Nua DB: search, create, upgrade.
Environment variables
setup_nua_db ¶
setup_nua_db()
Create the db if needed and also populate the configuration from both db values
and default parameters.
Module nua.orchestrator.nua_env¶
Managing os.environ for Nua variables.
Basically a wrapper for the self._env dict as a singleton, no class needed.
Module nua.orchestrator.persistent¶
Persistent
dataclass
¶
Management of persistent generated values (like passwords).
Persistent data is stored in the AppInstance. A Persistent object contains persitent
values for the AppInstance and its ressources. The ‘name’ attribute identifies the
provider owning the data, use ‘’ as provider_name for the data own by the
AppInstance itself.
Module nua.orchestrator.port_utils¶
ports_as_dict ¶
ports_as_dict(port_list)
Replace ports list by a dict with container port as key.
(Use str key because later conversion to json)
Module nua.orchestrator.provider¶
Provider ¶
Provider(provider_config)
providers
property
writable
¶
providers
List of sub providers of the object.
Warning: only AppInstance upper class has an actual use of ‘providers’.
This subclass Provider will always provide an empty list.
label_id
property
writable
¶
label_id
Sanitized version of the app label.
For providers, label_id is the label_id of the main app.
volume_declaration
property
writable
¶
volume_declaration
Docker volume declared on a non-container provider.
Thus, this volume needs to be started by the upper site.
post_run_status
property
¶
post_run_status
Return the image original nua-config ‘run/post-run-status’ value.
requires_network ¶
requires_network()
Heuristic to evaluate the need of docker private network.
Basic: using a docker container as provider probably implies need of network.
environment_ports ¶
environment_ports()
Return exposed ports and provider host (container name) as env variables.
To be used by remote container from same bridge network to connect to the
provider container port.
Module nua.orchestrator.provider_deps¶
Solve the order of evaluation of providers dynamic parameters.
ProviderDeps ¶
ProviderDeps()
Solve the order of evaluation of providers dynamic parameters.
Raise on circular dependencies.
Module nua.orchestrator.registry¶
Local docker registry.
WIP:
- at the moment, only use local docker configuratin of the host
- next step: configure a local registry, still using the ‘registry’ image,
but behind a nginx server with authentication and a managed directory
for storage.
Package nua.orchestrator.scripts¶
Module nua.orchestrator.scripts.deactivate_all_instances¶
Module nua.orchestrator.scripts.docker_list_all¶
Module nua.orchestrator.scripts.docker_remove_all¶
Module nua.orchestrator.scripts.list_instances¶
Module nua.orchestrator.scripts.mariadb_restore¶
Script to init the local mariadb password and access (for Nua user).
(Later: replace by flask ui access).
Module nua.orchestrator.scripts.print_backup¶
Module nua.orchestrator.scripts.print_deployments¶
Module nua.orchestrator.scripts.print_instances¶
Module nua.orchestrator.scripts.print_mounted_volumes¶
Module nua.orchestrator.scripts.print_used_ports¶
Module nua.orchestrator.scripts.reset_nginx_conf¶
Module nua.orchestrator.scripts.set_pg_pwd¶
Module nua.orchestrator.search_cmd¶
Nua : search image related funcitons.
image_available_locally ¶
image_available_locally(app_name)
Return True if image of app_name is available in local Docker daemon.
search_nua ¶
search_nua(app_name)
Search Nua image from the registries.
(local registry for now).
Return
list of path of local Nua archives sorted by version.
search_nua_print ¶
search_nua_print(app_name)
Search Nua image from the registries.
(local registry for now).
search_docker_tar_local ¶
search_docker_tar_local(app, tag)
Return list of path of local Nua archives sorted by version.
Module nua.orchestrator.state_journal¶
Class to to journalize state of the deployed apps.
StateJournal ¶
StateJournal()
Thin wrapper class to access the DeployConfig table of the Nua config database.
Use to store or retreive the active deployed configuration.
DeployConfig fields are
id = Column(Integer, primary_key=True, autoincrement=True)
previous = Column(Integer)
state = Column(String(16), default=INACTIVE)
created = Column(String(40))
modified = Column(String(40))
deployed = Column(JSON)
store_deployed_state ¶
store_deployed_state(deploy_config)
Store in the Nua DB the deployed state.
deploy_config = {“requested”: requested, “apps”: deepcopy(apps)}
restore_from_state_journal ¶
restore_from_state_journal(state)
Restore to the most recent deployment configuration that did succeed.
Package nua.orchestrator.util¶
Module nua.orchestrator.util.deep_access_dict¶
Module nua.orchestrator.util.deep_update¶
Module nua.orchestrator.utils¶
size_to_bytes ¶
size_to_bytes(size)
Convert string representing size to bytes value.
It uses basic regex to get results like: size_to_bytes(“2k”) 2048
size_to_bytes(“1MB”) 1048576
period_to_seconds ¶
period_to_seconds(period)
Convert human-like time period to seconds value.
It uses basic regex to get results like: period_to_seconds(“1h”)
3600 period_to_seconds(“24h”) 86400
Module nua.orchestrator.version¶
Module nua.orchestrator.volume¶
Volume ¶
Volume()
Representation of a volume attached to a container, either the main app container
or a Provider container.