Data Observability platform Help

Connections

Connections page is for managing all database connections.

  • Add new connections (by clicking "New connection" button)

  • Manage existing connections (change settings by clicking on a connection)

  • View and manage metadata for a connection, which is used by most DQM functionalities.

  • Check connection status (update status by clicking on status icon)

Adding a new connection

  1. On Connections page click "New connection" button:

  2. Insert database connection information:

ConnectionsAdd
  • Driver/Connector – database JDBC driver (see chapter Connector settings to add new drivers)

  • Connection name – custom name for the connection

  • Server – database server address

  • Port – database port

  • Database name – database name in server

  • Properties – additional connection properties

  • User – database user (we recommend using a user with read-only rights)

  • Password – password for the user

  • Parallel queries – maximum parallel connections to database for a user

  • Query timeout (s) – maximum number of seconds the connection waits for a response from the database

  1. Click "Add connection" button

  2. New connection will appear on the list and connection status will show if connection was successful

Adding additional database users

The platform supports configuring additional database users to be used for querying results and seeing results of past executions. The use case for this is if you want to limit some users of the platform to not have access to data that is reachable with the master user configured for the connection.

Additional database users can be configured for existing connections through a menu accessible via the settings button on the right of a connection's master user's username.

ConnectionsAdditionalUsersButton.png

Each additional database user will apply for a selection of user roles. The additional database users are ordered, meaning that for querying data, the database user with the highest rank will be chosen that is applicable for the user's roles.

ConnectionsAdditionalUsers.png

In the example above, if a user has the roles Analyst and Developer, then their queries through the platform will be executed using the database user postgres. If another user has roles steward and maintenance, then for them the database user analyst will be used.

If no match is found from the additional users list, then the master user of the connection will be used for executions.

The database users can be re-ordered by holding and dragging the handle on right of the edit button.

Example - Add a Connection

Example connections

Connections example spark
Connections example redshift
Connections example snowflake
Example sharepoint
Connections example excel
Connections example csv
Connections example custom

Deleting an existing connection

  1. Click on the connection you would like to delete

  2. From opened window click "Delete connection" button

Metadata

Metadata is used to automatically create test cases, generate SQLs and manage Profiling. Metadata can be managed by clicking on a specific connection:

Adding Metadata

  • Select schemas or insert them manually (comma separated):

    AddMetaData.png

  • Click "+" button to import schemas

  • Metadata should now be available

    connectionsMetadataList
    If the "Update" flag is enabled for a schema, then additions and changes to objects and columns are automatically synced every 120 minutes (can be changed in Settings -> General.

    If the "Delete" flag is enabled for a schema, then object and column removals from the database are automatically synced.

    Metadata can also be manually updated or deleted by selecting schemas and clicking "Update selected" or "Delete selected" button.

Disabling objects

Sometimes we need to disable metadata loading for certain objects in the schema to avoid sensitive metadata being usable for all users

  • Hover over the schema which objects you wish to exclude

  • Click on the connectionDisableIcon icon

  • Choose all objects you wish to Exclude/Include (You can use Shift+Click to choose multiple checkboxes)

    connectionMetaDataExclude

    NB! When you re-include an excluded object, metadata will be loaded again for given schema

File storage

Users can upload CSV files directly to application internal storage for easier CSV access. Uploaded files are encrypted at rest.

Uploading a file

From side menu users can access connections-file-storage button

connections-files

Files can be uploaded/downloaded from the same view with correct privileges. Select the folder where the file will be uploaded.

Creating and editing folder

  • New folders can be created by clicking the connections-plus icon while hovering All files folder. Folders and files can also be dragged to a new location.

  • To edit or delete existing folder hover on folder connections-file-storage-edit-folder and click pencil icon.

Using stored files in CSV connection

  1. Set up a new CSV connection

connections-local-csv
  • Connector – Choose CSV connector

  • Connection name – Any custom name for CSV group

  • Storage/Path – Location for CSV files

  • Properties – Default separator for CSV is "," which can be changed based input files

Last modified: 29 January 2025