Administrator Guide

• 1: Clusters
• 1.1: View Cluster Details
• 1.2: Cluster Actions
• 1.2.1: Upgrade Cluster
• 1.2.2: Rescale Cluster
• 1.2.3: Stop and Start a Cluster
• 1.2.4: Export Cluster Settings
• 1.2.5: Replicate a Cluster
• 1.2.6: Destroy Cluster
• 1.3: Cluster Settings
• 1.4: Configure Cluster
• 1.4.1: How to Configure Cluster Settings
• 1.4.2: How to Configure Cluster Profiles
• 1.4.3: How to Configure Cluster Users
• 1.5: Launch New Cluster
• 1.6: Cluster Alerts
• 1.7: Cluster Health Check
• 1.8: Cluster Monitoring
• 1.9: Cluster Logs
• 2: Access Control
• 2.1: Role Based Access and Security Tiers
• 2.2: Account Management

1 - Clusters

ClickHouse databases are managed through clusters, which harness the power of distributed processing to quickly deliver results on even the most complex and data intensive queries.

Altinity.Cloud users can create their own ClickHouse Clusters tailored to their organization’s needs.

1.1 - View Cluster Details

Cluster Details Page

Once a cluster has been launched, its current operating details can be viewed by selecting the cluster from the Clusters View. This displays the Cluster Details page. From the Cluster Dashboard page, select Nodes to view the Nodes Summary page.

From the Cluster Details page, users can perform the following:

• A: Manage the cluster’s:
  • Actions
  • Configuration
  • Tables and structure with Explore
  • Alerts
  • View the Cluster’s Logs
• B: Check Cluster Health.
• C: View the cluster’s Access Point.
• D: Monitor the Cluster and its Queries.
• E: View summary details for the Cluster or Node. Select Nodes to view details on the cluster’s Nodes.

Nodes Summary

The Nodes Summary Page displays all nodes that are part of the selected cluster. From this page the following options and information is available:

1. The Node Summary that lists:
   1. Endpoint: The connection settings for this node. See Node Connection.
   2. Details and Node View: Links to the Node Dashboard and Node Metrics.
   3. Version: The ClickHouse version running on this node.
   4. Type: The processor setting for the node.
   5. Node Storage: Storage space in GB available.
   6. Memory: RAM memory allocated for the node.
   7. Availability Zone: Which AWS Availability Zone the node is hosted on.

• Select Node View or View for the specific node to access the Node DashboardNode Metrics, and Node Schema sections.

Node Connection

The Node Connection Details shows how to connect from various clients, including the clickhouse-client, JDBC drivers, HTTPS, and Python. Unlike the Cluster Access Point, this allows a connection directly to the specific node.

Node Dashboard

From the Node Dashboard Page users can:

• A: Manage the node’s:
  • Tables and structure with Explore
  • View the Node’s Logs
• B: Check the node’s health.
• C: View summary details node, it’s Metrics and its Schema.
• D: Perform Node Actions.

Node Metrics

Node Metrics provides a breakdown of the node’s performance, such as CPU data, active threads, etc.

Node Schema

The Node Schema provides a view of the databases’ schema and tables. For more information on how to interact with a Node by submitting queries, viewing the schema of its databases and tables, and viewing process, see the Cluster Explore Guide.

Node Actions

The following actions can be performed on a Node:

• Delete: Removes the Node from the cluster.

1.2 - Cluster Actions

Launched clusters can be have different actions applied to them based on your needs.

1.2.1 - Upgrade Cluster

Clusters can be upgraded to versions of ClickHouse other than the one your cluster is running.

When upgrading to a ClickHouse Altinity Stable Build, review the release notes for the version that you are upgrading to.

How to Upgrade an Altinity Cloud Cluster

To upgrade a launched cluster:

1. Select Actions for the cluster to upgrade.

2. Select Upgrade.

3. Select the ClickHouse version to upgrade to.

4. Select Upgrade to start the process.

   

The upgrade process completion time varies with the size of the cluster, as each server is upgraded individually. This may cause downtime while the cluster is upgraded.

1.2.2 - Rescale Cluster

The size and structure of the cluster may need to be altered after launching based on your organization’s needs. The following settings can be rescaled:

• Number of Shards
• Number of Replicas
• Node Type
• Node Storage
• Number of Volumes
• Apply to new nodes only: This setting will only effect nodes created from this point forward.

See Cluster Settings for more information.

How to Rescale a Cluster

To rescale a cluster:

1. Select Actions for the cluster to rescale.

2. Select Rescale.

3. Set the new values of the cluster.

4. Click OK to begin rescaling.

   

Depending on the size of the cluster, this may take several minutes.

1.2.3 - Stop and Start a Cluster

To stop an launched cluster, or start a stopped cluster:

1. From either the Clusters View or the Cluster Details Page, select Actions.
   1. If the cluster is currently running, select Stop to halt its operations.
   2. If the cluster has been stopped, select Start to restart it.

Depending on the size of your cluster, it may take a few minutes until it is fully stopped or is restarted. To access the health and availability of the cluster, see Cluster Health or the Cluster Availability.

1.2.4 - Export Cluster Settings

The structure of an Altinity Cloud cluster can be exported as JSON. For details on the cluster’s settings that are exported, see Cluster Settings.

To export a cluster’s settings to JSON:

1. From either the Clusters View or the Cluster Details Page, select Actions, then select Export.
2. A new browser window will open with the settings for the cluster in JSON.

1.2.5 - Replicate a Cluster

Clusters can be replicated with the same or different settings. These can include the same database schema as the replicated cluster, or launched without the schema. This may be useful to create a test cluster, then launch the production cluster with different settings ready for production data.

For complete details on Altinity.Cloud clusters settings, see Cluster Settings.

To create a replica of an existing cluster:

1. From either the Clusters View or the Cluster Details Page, select Actions, then select Launch a Replica Cluster.
2. Enter the desired values for Resources Configuration.

   1. To replicate the schema of the source directory, select Replicate Schema.

      

   2. Click Next to continue.

3. High Availability Configuration, and Connection Configuration.
   1. Each section must be completed in its entirety before moving on to the next one.
4. In the module Review & Launch, verify the settings are correct. When finished, select Launch.

Depending on the size of the new cluster it will be available within a few minutes. To verify the health and availability of the new cluster, see Cluster Health or the Cluster Availability.

1.2.6 - Destroy Cluster

When a cluster is no longer required, the entire cluster and all of its data can be destroyed.

• IMPORTANT NOTE: Once destroyed, a cluster can not be recovered. It must be manually recreated.

To destroy a cluster:

1. From either the Clusters View or the Cluster Details Page, select Actions, then select Destroy.

2. Enter the cluster name, then select OK to confirm its deletion.

   

1.3 - Cluster Settings

ClickHouse Clusters hosted on Altinity.Cloud have the following structural attributes. These determine options such as the version of ClickHouse installed on them, how many replicas, and other important features.

1.4 - Configure Cluster

Once a cluster has been launched, it’s configuration can be updated to best match your needs.

1.4.1 - How to Configure Cluster Settings

Cluster settings can be updated from the Clusters View or from the Cluster Details by selecting Configure > Settings.

• IMPORTANT NOTE: Changing a cluster’s settings will require a restart of the entire cluster.

Note that some settings are locked - their values can not be changed from this screen.

How to Set Troubleshooting Mode

Troubleshooting mode prevents your cluster from auto-starting after a crash. To update this setting:

1. Toggle Troubleshooting Mode either On or Off.

How to Edit an Existing Setting

To edit an existing setting:

1. Select the menu on the left side of the setting to update.
2. Select Edit.
3. Set the following:
   1. Setting Type.
   2. Name
   3. Value
4. Select OK to save the setting.

How to Add a New Setting

To add a new setting to your cluster:

1. Select Add Setting.
2. Set the following:
   1. Setting Type.
   2. Name
   3. Value
3. Select OK to save the setting.

How to Delete an Existing Setting

To delete an existing setting:

1. Select the menu on the left side of the setting to update.
2. Select OK.
3. Select Remove to confirm removing the setting.

1.4.2 - How to Configure Cluster Profiles

Cluster profiles allow you to set the user permissions and settings based on their assigned profile.

The Cluster Profiles can be accessed from the Clusters View or from the Cluster Details by selecting Configure > Settings.

Add a New Profile

To add a new cluster profile:

1. From the Cluster Profile View page, select Add Profile.
2. Provide profile’s Name and Description, then click OK.

Edit an Existing Profile

To edit an existing profile:

1. Select the menu to the left of the profile to update and select Edit, or select Edit Settings.
2. To add a profile setting, select Add Setting and enter the Name and Value, then click OK to store your setting value.
3. To edit an existing setting, select the menu to the left of the setting to update. Update the Name and Value, then click OK to store the new value.

Delete an Existing Profile

To delete an existing profile:

1. Select the menu to the left of the profile to update and select Delete.
2. Select OK to confirm the profile deletion.

1.4.3 - How to Configure Cluster Users

The cluster’s Users allow you to set one or more entities who can access your cluster, based on their Cluster Profile.

Cluster users can be updated from the Clusters View or from the Cluster Details by selecting Configure > Users.

How to Add a New User

To add a new user to your cluster:

1. Select Add User

2. Enter the following:

   

   1. Login: the name of the new user.
   2. Password and Confirm Password: the authenticating credentials for the user. Th
   3. Networks: The networks that the user can connect from. Leave as 0.0.0.0/0 to allow access from all networks.
   4. Databases: Which databases the user can connect to. Leave empty to allow access all databases.
   5. Profile: Which profile settings to apply to this user.

3. Select OK to save the new user.

How to Edit a User

To edit an existing user:

1. Select the menu to the left of the user to edit, then select Edit.
2. Enter the following:
   1. Login: the new name of the user.
   2. Password and Confirm Password: the authenticating credentials for the user. Th
   3. Networks: The networks that the user can connect from. Leave as 0.0.0.0/0 to allow access from all networks.
   4. Databases: Which databases the user can connect to. Leave empty to allow access all databases.
   5. Profile: Which profile settings to apply to this user.
3. Select OK to save the updated user user.

How to Delete a User

1. Select the menu to the left of the user to edit, then select Delete.
2. Select OK to verify the user deletion.

1.5 - Launch New Cluster

Launching a new ClickHouse Cluster is incredibly easy, and only takes a few minutes. For those looking to create their first ClickHouse cluster with the minimal steps, see the Quick Start Guide. For complete details on Altinity.Cloud clusters settings, see Cluster Settings.

To launch a new ClickHouse cluster:

1. From the Clusters View page, select Launch Cluster. This starts the Cluster Launch Wizard.

   

2. Enter the desired values for Resources Configuration, High Availability Configuration, and Connection Configuration.

   1. Each section must be completed in its entirety before moving on to the next one.

3. In the module Review & Launch, verify the settings are correct. When finished, select Launch.

Within a few minutes, the new cluster will be ready for your use and display that all health checks have been passed.

1.6 - Cluster Alerts

The Cluster Alerts module allows users to set up when they are notified for a set fo events. Alerts can either be a popup, displaying the alert when the user is logged into Altinity.Cloud, or email so they can receive an alert even when they are not logged into Altinty.Cloud.

To set which alerts you receive:

1. From the Clusters view, select the cluster to for alerts.

2. Select Alerts.

   

3. Add the Email address to send alerts to.

4. Select whether to receive a Popup or Email alert for the following events:

   1. ClickHouse Version Upgrade: Alert triggered when the version of ClickHouse that is installed in the cluster has a new update.
   2. Cluster Rescale: Alert triggered when the cluster is rescaled, such as new shards added.
   3. Cluster Stop: Alert triggered when some event has caused the cluster to no longer running.
   4. Cluster Resume: Alert triggered when a cluster that was stopped has resumed operations.

1.7 - Cluster Health Check

From the Clusters View, you can see the health status of your cluster and its nodes at a glance.

How to Check Node Health

The quick health check of your cluster’s nodes is displayed from the Clusters View. Next to the cluster name is a summary of your nodes’ statuses, indicating the total number of nodes and how many nodes are available.

How to Check Cluster Health

The overall health of the cluster is shown in the Health row of the cluster summary, showing the number of health checks passed.

Click checked passed to view a detailed view of the cluster’s health.

How to View a Cluster’s Health Checks

The cluster’s Health Check module displays the status of the following health checks:

• Access point availability check
• Distributed query check
• Zookeeper availability check
• Zookeeper contents check
• Readonly replica check
• Delayed inserts check

To view details on what queries are used to verify the health check, select the caret for each health check.

1.8 - Cluster Monitoring

Altinity.Cloud integrates Grafana into its monitoring tools. From a cluster, you can quickly access the following monitoring views:

• Cluster Metrics
• Queries
• Logs

How to Access Cluster Metrics

To access the metrics views for your cluster:

1. From the Clusters view, select the cluster to monitor.
2. From Monitoring, select the drop down View in Grafana and select from one of the following options:
   1. Cluster Metrics
   2. Queries
   3. Logs
3. Each metric view opens in a separate tab.

Cluster Metrics

Cluster Metrics displays how the cluster is performing from a hardware and connection standpoint.

Some of the metrics displayed here include:

• DNS and Distributed Connection Errors: Displays the rate of any connection issues.
• Select Queries: The number of select queries submitted to the cluster.
• Zookeeper Transactions: The communications between the zookeeper nodes.
• ClickHouse Data Size on Disk: The total amount of data the ClickHouse database is using.

Queries

The Queries monitoring page displays the performance of clusters, including the top requests, queries that require the most memory, and other benchmarks. This can be useful in identifying queries that can cause performance issues and refactoring them to be more efficient.

Log Metrics

The Log monitoring page displays the logs for your clusters, and allows you to make queries directly on them. If there’s a specific detail you’re trying to iron out, the logs are the most granular way of tracking down those issues.

1.9 - Cluster Logs

Altinity.Cloud provides the cluster log details so users can trace down specific issues or performance bottlenecks.

To access a cluster’s logs:

1. From the Clusters view, select the cluster to for alerts.
2. Select Logs.
3. From the Log Page, you can display the number of rows to view, or filter logs by specific text.
4. To download the logs, select the download icon in the upper right corner (A).
5. To refresh the logs page, select the refresh icon (B).

The following logs are available:

• ACM Logs: These logs are specific to Altinity.Cloud issues and include the following:
  • System Log: Details the system actions such as starting a cluster, updating endpoints, and other details.
  • API Log: Displays updates to the API and activities.
• ClickHouse Logs: Displays the Common Log that stores ClickHouse related events. From this view a specific host can be selected form the dropdown box.
• Backup Logs: Displays backup events from the clickhouse-backup service. Log details per cluster host can be selected from the dropdown box.
• Operator Logs: Displays logs from the Altinity Kubernetes Operator service, which is used to manage cluster replication cluster and communications in the Kubernetes environment.

2 - Access Control

Altinity.Cloud provides role based access control. Depending the role granted to an Altinity.Cloud Account, they can assign other Altinity.Cloud accounts roles and grant permissions to access organizations, environments, or clusters.

2.1 - Role Based Access and Security Tiers

Access to ClickHouse data hosted in Altinity.Cloud is controlled through a combination of security tiers and account roles. This allows companies to tailor access to data in a way that maximizes security while still allowing ease of access.

Security Tiers

Altinity.Cloud groups sets of clusters together in ways that allows companies to provide Accounts access only to the clusters or groups of clusters that they need to.

Altinity.Cloud groups clusters into the following security related tiers:

• Nodes: The most basic level - an individual ClickHouse database and tables.
• Clusters: These contain one or more nodes provide ClickHouse database access.
• Environments: Environments contain one or more clusters.
• Organizations: Organizations contain one or more environments.

Account access is controlled by assigning an account a single role and a security tier depending on their role. A single account can be assigned to multiple organizations, environments, multiple clusters in an environment, or a single cluster depending on their account role.

Account Roles

The actions that can be taken by Altinity.Cloud accounts is based on the role they are assigned. The following roles and their actions based on the security tier is detailed in the table below:

The account roles are tied into the security tiers, and allow an account to access multiple environment and clusters depending on what type of tier they are assigned to.

For example, we may have the following situation:

• Accounts peter, paul, and mary and jessica are all members of the organization HappyDragon.
• HappyDragon has the following environments: HappyDragon_Dev and HappyDragon_Prod, each with the clusters marketing, sales, and ops.

The accounts are assigned the following roles and security tiers:

In this scenario, mary has the ability to access the environment HappyDragon_Prod, or can create new environments and manage them and any clusters within them. However, she is not able to edit or access HappyDragon_Dev or any of its clusters.

• Both peter and jessica have the ability to create and remove clusters within their assigned environments.
  • peter is able to modify the clusters in the environment HappyDragon_Dev.
  • jessica can modify clusters in both environments.
• paul can only access the cluster marketing in the environment HappyDragon_Prod.

2.2 - Account Management

Altinity.Cloud accounts with the role orgadmin are able to create new Altinity.Cloud accounts and associate them with organizations, environments, and one or more clusters depending on their role. For more information on roles, see Role Based Access and Security Tiers.

Account Page

The Account Page displays all accounts assigned to the same Organization and Environments as the logged in account.

For example: the accounts mario, luigi, and peach and todd are members of the organizations MushroomFactory and BeanFactory as follows:

• peach will be able to see their account and todd in the Account Page, while accounts mario and luigi will be hidden from them.
• mario will be able to see their account and luigi.

Access Accounts

To access the accounts that are assigned to the same Organizations and Environments as the logged in user with the account role orgadmin:

1. Login to Altinity.Cloud with an account granted the orgadmin role.
2. From the left navigation panel, select Accounts.
3. All accounts that are in the same Organizations and Environments as the logged in account will be displayed.

Account Details

Accounts have the following details that can be set by an account with the orgadmin role:

1. Common Information:
   1. Name: The name of the account.
   2. Email (Required): The email address of the account. This will be used to login, reset passwords, notifications, and other uses. This must be a working email for these functions to work.
   3. Password: The password for the account. Once a user has authenticated to the account, they can change their password.
   4. Confirm Password: Confirm the password for the account.
   5. Role (Required): The role assigned to the account. For more information on roles, see Role Based Access and Security Tiers.
   6. Organization: The organization assigned to the account. Note that the orgadmin can only assign accounts the same organizations that the orgadmin account also belongs to.
   7. Suspended: When enabled, this prevents the account from logging into Altinity.Cloud.
2. Environment Access:
   1. Select the environments that the account will require access to. Note that the orgadmin can only assign accounts the same environments that the orgadmin account also belongs to.
3. Cluster Access:
   1. This is only visible if the Role is set to envuser. This allows one or more clusters in the environments the new account was assigned to in Environmental Access to be accessed by them.
4. API Access:
   1. Allows the new account to make API calls from the listed domain names.

Account Actions

Create Account

orgadmin accounts can create new accounts and assign them to the same organization and environments they are assigned to. For example, continuing the scenario from above, if account peach is assigned to the organization MushroomFactory and the environments MushroomProd and MushroomDev, they can assign new accounts to the organization MushroomFactory, and to the environments MushroomProd or MushroomDev or both.

To create a new account:

1. Login to Altinity.Cloud with an account granted the orgadmin role.

2. From the left navigation panel, select Accounts.

3. Select Add Account.

4. Set the fields as listed in the Account Details section.

   

5. Once all settings are completed, select Save. The account will be able to login with the username and password, or if their email address is registered through Google, Auth0.

Edit Account

1. Login to Altinity.Cloud with an account granted the orgadmin role.
2. From the left navigation panel, select Accounts.
3. From the left hand side of the Accounts table, select the menu icon for the account to update and select Edit.
4. Update the fields as listed in the Account Details section.
5. When finished, select Save.

Suspend Account

Instead of deleting an account, setting an account to Suspended may be more efficient to preserve the accounts name and other settings. A suspended account is unable to login to Altinity.Cloud. This includes directly logging through a browser and API calls made under the account.

To suspend or activate an account:

1. Login to Altinity.Cloud with an account granted the orgadmin role.
2. From the left navigation panel, select Accounts.
3. From the left hand side of the Accounts table, select the menu icon for the account to update and select Edit.
   1. To suspend an account, toggle Suspended to on.
   2. To activate a suspended account, toggle Suspended to off.
4. When finished, select Save.

Delete Account

Accounts can be deleted which removes all information on the account. Clusters and environments created by account will remain.

To delete an existing account:

1. Login to Altinity.Cloud with an account granted the orgadmin role.
2. From the left navigation panel, select Accounts.
3. From the left hand side of the Accounts table, select the menu icon for the account to update and select Delete.
4. Verify the account is to be deleted by selecting OK.