Release Notes 2.1

What's New

Version 2.1 includes following new features and changes.

Fulltext Search

The index that was used for full-text search in older versions was prone to break on very big datasets. In Version 2.1 there is a new way how full-text search indexes are created and queried. Future releases will include further performance improvements of the search.

caution

Existing data has to be reprocessed to make them available in the search.

Performance Improvement of "Merge" Data Import

The performance of data imports using "Merge" load type was improved drastically. While importing data for an entity with 300.000 nodes and 15 million relationships took 2 hours and 38 minutes with the old logic, the same entity now only takes 6 minutes to import.

GBS Dashboard

Version 2.1 introduces a new dashboard in the Graph Builder Services. The data previously available on the dashboard was split into different views. Additionally, an overview of the whole system was added to the dashboard.

Telemetry

OpenTelemetry, Jaeger and Prometheus are added to the stack and integrated in services to trace requests across the stack. This helps to track usage of components and identify bottlenecks in communication between services.

New Way of Storing Relationships

When importing data, all corresponding relationships are precalculated and stored in the system. The way how these relationships are stored has been changed which results in a performance improvement. In the past it could take a long time to rename a property that is part of a relationship (~4 minutes for 15 million relationships). With the new approach the same rename only takes ~2 minutes and 30 seconds. In the course of these changes obsolete artifact types " PartID" and "RelatedNodePath" were removed.

Improvements

• [Frontend] Consolidated filtering and removing nodes in sandbox
• [Frontend] Performance improvement when retrieving nodes in a Data Lab
• [Frontend] Improved error messages
• [Frontend] Filter button is permanently visible in top navigation bar
• [Frontend] Cleaned up of left navigation bar
• [Frontend] Added new options for context menu
• [Frontend] Allow easy reset to defaults of different personalization settings
• [Frontend] Result table in add related and search menu allows to select different properties to inspect data
• [Frontend] It is possible to personalize icons for labels
• [Frontend] Consolidated left menu in sandbox mode with menu in exploration mode
• [Frontend] Settings button was added to node context menu to easily access context menu settings
• [Frontend] "Property selection" window was moved from the right menu bar to a dialog opened through setting button in properties panel
• [Frontend] In "Filter by property" the global "Keyword" search was replaced by "Search all properties"
• [Frontend] Filter history now includes what was filtered for
• [Frontend] Active filters are displayed as tags to the filter menu
• [Frontend] Webvis library was updated to work with 3DHub 3.6
• [Frontend] Performance improvements by replacing calls to the now deprecated /api/graph/query/nodes endpoint
• [GBS] List of artifact definitions in target entities now also displays the artifact type [GBS] A warning will be displayed in data importer when selecting a target entity whose connected source entity is missing
• [GBS] Added explanation about order for discrete values
• [GBS] When a property, that is part of a relationship, is deleted, the corresponding relationship is also deleted
• [GBS] When creating an entity from a container, excluded properties are not shown in builders anymore
• [GBS] Separators are added to large numbers
• [GBS] System configuration now requires admin role
• [GBS] In Access Rules, when selecting an already added entity, already added properties will automatically be selected
• [GBS] "DB Query" and "Graph Query" were removed for security reasons
• [GBS] Future dates cannot be selected in Queue Jobs or Logs anymore

Bug Fixes

• [Frontend] Open new session button in the Data Lab does not open a new tab
• [Frontend] When switching tabs, correlations in Data Lab are scaled down
• [Frontend] After applying a scope, the toggle is malfunctioning
• [Frontend] Discrete values are not correctly handled in Data Lab charts
• [Frontend] Date and range filter are malfunctioning
• [Frontend] Adding a rule group to a scope breaks the layout
• [Frontend] Filter and select is not working for data with whitespaces or hyphens
• [GBS] Opening the data table of an entity breaks layouts in other GBS views
• [GBS] Properties used in Access Rules cannot be deleted
• [GBS] Masked values in System Configuration cannot be edited
• [GBS] Clicking "Create new entities" in Data Pump container twice, shows empty list
• [GBS] Renaming an entity with properties ending on _ID, created superfluous properties
• [GBS] Fixed multiple problems when creating entities from container
• [GBS] Renaming a property that is used in multiple relationships causes an error
• [GBS] Creating a relationship without selecting a property but typing its name leads to broken relationship

Deprecations

• [EBS] /api/graph/query/nodes endpoint will be replaced by specific endpoints in future versions and should not be used anymore. For retrieving artifacts /api/artifacts/per-nodeid endpoint was added. Related nodes can be retrieved via already existing endpoints.

Deployment Changes

• OpenTelemetry, Jaeger and Prometheus containers were added to enable tracing and metrics of communication between services in the stack.
• All containers in the stack now run in non-root mode

Software Updates

3^rd party software was updated to the following versions:

Name	Version
Neo4j	5.5
Apache Airflow	2.6
Keycloak	21
OpenTelemetry	0.76
Jaeger	1.42
Prometheus	2.42