Data Observability platform Help

Connections

Connections page is for managing all database connections.

  • Add new connections (by clicking "New connection" button)

  • Manage existing connections (change settings by clicking on a connection)

  • View and manage metadata for a connection, which is used by most DQM functionalities.

  • Check connection status (update status by clicking on status icon)

Adding a new connection

  1. On Connections page click "New connection" button:

  2. Insert database connection information:

ConnectionsAdd
  • Driver/Connector – database JDBC driver (see chapter Connector settings to add new drivers)

  • Connection name – custom name for the connection

  • Server – database server address

  • Port – database port

  • Database name – database name in server

  • Properties – additional connection properties

  • User – database user (we recommend using a user with read-only rights)

  • Password – password for the user

  • Parallel queries – maximum parallel connections to database for a user

  • Query timeout (s) – maximum number of seconds the connection waits for a response from the database

  1. Click "Add connection" button

  2. New connection will appear on the list and connection status will show if connection was successful

Example - Add a Connection

Example connections

Connections example spark
Connections example redshift
Connections example snowflake
Connections example excel
Connections example csv
Connections example custom

Deleting an existing connection

  1. Click on the connection you would like to delete

  2. From opened window click "Delete connection" button

Metadata

Metadata is used to automatically create test cases, generate SQLs and manage Profiling. Metadata can be managed by clicking on a specific connection:

Adding Metadata

  • Select schemas or insert them manually (comma separated):

    AddMetaData.png

  • Click "+" button to import schemas

  • Metadata should now be available

    connectionsMetadataList
    If the "Update" flag is enabled for a schema, then additions and changes to objects and columns are automatically synced every 120 minutes (can be changed in Settings -> General.

    If the "Delete" flag is enabled for a schema, then object and column removals from the database are automatically synced.

    Metadata can also be manually updated or deleted by selecting schemas and clicking "Update selected" or "Delete selected" button.

Disabling objects

Sometimes we need to disable metadata loading for certain objects in the schema to avoid sensitive metadata being usable for all users

  • Hover over the schema which objects you wish to exclude

  • Click on the connectionDisableIcon icon

  • Choose all objects you wish to Exclude/Include (You can use Shift+Click to choose multiple checkboxes)

    connectionMetaDataExclude

    NB! When you re-include an excluded object, metadata will be loaded again for given schema

File storage

Users can upload CSV files directly to application internal storage for easier CSV access. Uploaded files are encrypted at rest.

Uploading a file

From side menu users can access connections-file-storage button

connections-files

Files can be uploaded/downloaded from the same view with correct privileges. Select the folder where the file will be uploaded.

Creating and editing folder

  • New folders can be created by clicking the connections-plus icon while hovering All files folder. Folders and files can also be dragged to a new location.

  • To edit or delete existing folder hover on folder connections-file-storage-edit-folder and click pencil icon.

Using stored files in CSV connection

  1. Set up a new CSV connection

connections-local-csv
  • Connector – Choose CSV connector

  • Connection name – Any custom name for CSV group

  • Storage/Path – Location for CSV files

  • Properties – Default separator for CSV is "," which can be changed based input files

Last modified: 29 November 2024