Data Catalog

Enhance data discoverability

Manage metadata to facilitate data searchability and improve discoverability, reducing friction for analytics engineers.
Features

Helping you elevate your data
management experience

Simplify Data Analysis with Trocco’s Data Catalog

Trocco® transforms metadata management with a user-friendly data catalog feature, automating metadata acquisition for easier data location and comprehension.
This streamlines the analysis process, speeding up insight generation and making data analysis accessible to users of varying skill levels, thereby enhancing the overall efficiency of analytics infrastructures.

Effortless Metadata Accumulation as Your Data Platform Evolves

Trocco's data catalog autonomously grows with your analytics platform, automatically accumulating metadata from diverse sources and warehouses, eliminating manual entry.
This feature adapts to data transfer settings and expansions, significantly reducing metadata management effort and evolving as a key asset to enhance the analytics infrastructure.

Intuitive Tools for Comprehensive Metadata Insight

Trocco's table overview screen provides easy access to table and column metadata, enriching data handling with advanced preview functionalities like summary statistics, and filtering and sorting capabilities.
This enhances data comprehension, allows deep dives into records, and streamlines data utilization, offering a comprehensive understanding of data distribution and overview.

Advanced Query Editor for Engineers

Masking
Hashing (SHA256)
Data type conversion
Programming ETL (Ruby / Python)
String conversion (NFKC)
Record filter
String substitution (regular expression)
Trocco's query editor enhances engineer productivity by offering instant query writing from any data catalog screen, with features like auto-completion, metadata display, query saving, execution previews, and CSV export.
This multifunctional tool is tailored for convenience and efficiency, matching the capabilities of daily-use engineering editors.

Technical Capabilities

Elevate Your Data Workflow and Streamline Analytics with Automated Schema, Custom Templates, and Dynamic Variables

Utilize the Search Function
Trocco® leverages Elasticsearch for fuzzy search, enhancing data discoverability for novices by simplifying navigation through vast data landscapes, making data location effortless.
Create Column Lineage Visualization
With a detailed genealogy diagram, users can visually trace the origin and application of each column in a table, understanding the impact and lineage of data elements. This aids in comprehending the data's journey and its transformations.
Get ER Diagram Accessibility
Trocco  provides instant clarity on the relationships between tables, essential for JOIN operations. Even with fully normalized tables, users can quickly grasp how tables relate to one another, facilitating more efficient data modeling and querying.
Perform JOIN Analysis
The platform suggests JOIN keys for all scenarios, aided by tools like Venn diagrams, summary statistics, and table previews, helping users identify optimal JOIN strategies for data analysis.
Easy Manual Metadata Entry
Trocco® supports markdown for metadata input, enabling standardized, templated metadata management for comprehensive, consistent documentation, improving data governance and understanding.
Initiate Metadata Transfer
Trocco's metadata transfer capability underscores seamless data management, enabling metadata migration and synchronization across the infrastructure, ensuring data consistency and insight accessibility.
See other functions

Unlock Comprehensive Data Management: See What else Trocco offers to you

Workflow
(job management)

Complex pipelines are also defined in the GUI. There are plenty of connections necessary for data engineers
See More Details

Data Mart
Generation (ELT)

Just write the conversion process in the SQL editor, and the logs on DWH are easy to use for business.
See More Details

Data Catalog

Eliminate bottlenecks from data understanding to use, and shorten the lead time to problem solving
See More Details

Data check

Flexible testing and verification of data accumulated on DWH is possible.
See More Details

Git and API linkage function

Infrastructure as Code as a data analysis platform. Enhanced API cooperation with external tools
See More Details

Operational support (DevOps)

Development of support functions necessary for the operation of data analysis platforms, such as notifications and re-execution, from the engineer's perspective
See More Details

Web behavior log
collection SDK

Collect website log events with a dedicated SDK and transfer them to your favorite DWH
See More Details

Security

Equipped with functions such as encryption and IP restriction to handle important data
See More Details

Support

A dedicated customer will be in charge and support the response and operation of failures.
See More Details

Data Mart Generation (ELT)

Just write the conversion process in the SQL editor, and the logs on DWH are easy to use for business.
See More Details

Data Mart Generation (ELT)

This feature enables the transformation of data warehouse (DWH) logs into a business-friendly format by simply using an SQL editor for the conversion process. It streamlines the preparation of data marts, making it easier for businesses to analyze and draw insights from their data efficiently.

See More Details

Workflow (job management)

Complex pipelines are also defined in the GUI. There are plenty of connections necessary for data engineers
See More Details

Workflow (job management)

Users can define complex data pipelines through an easy to use graphical user interface (GUI), facilitating easier management of data flows. It supports numerous connector options, making it a versatile tool for data engineers who need to manage and integrate diverse data sources and processes.Scheduled DWH Queries support the regular execution of queries with customizable variables and Slack notifications to keep teams updated. Furthermore, Job Dependencies integrate with workflow functionality to create dependencies for systematic data aggregation post data transfers, optimizing data processing workflows. This comprehensive approach ensures efficient and effective data management and analysis.

See More Details

Data Catalog

Eliminate bottlenecks from data understanding to use, and shorten the lead time to problem solving
See More Details

Data Catalog

This function addresses the challenge of navigating and understanding large datasets by eliminating bottlenecks that slow down the transition from data comprehension to utilization. It effectively reduces the time needed to identify and resolve data-related issues, enhancing overall productivity and decision-making speed.Enhanced table previews offer detailed metadata, summary statistics for data distribution insights, and functionalities like filtering and sorting, streamlining data understanding.The query editor caters to engineers with features like auto-completion, query saving, execution previews, and CSV export options, enabling immediate query writing from anywhere within the data catalog.

See More Details

Data check

Flexible testing and verification of data accumulated on DWH is possible.
See More Details

Data check

Offers the capability for flexible testing and verification of data stored in the data warehouse. This feature ensures the integrity and accuracy of data before it's used for analysis, reporting, or decision-making, contributing to the reliability of data-driven insights for your business. By integrating these checks with workflow functionality, the entire process of data verification and integration is automated, streamlining the monitoring and maintenance of data quality and consistency across platforms.

See More Details

Git and API linkage function

Infrastructure as Code as a data analysis platform. Enhanced API cooperation with external tools
See More Details

Git and API linkage function

This extends the principles of Infrastructure as Code (IaC) to the realm of data analysis platforms, promoting more efficient and reliable setup and management of data infrastructure. By linking data transfer settings with Git in YAML format, users can achieve version control for their configurations, allowing for changes to be reviewed through pull requests by the team. Furthermore, API integration facilitates job execution and status monitoring, with the capability to specify custom variable settings at runtime. It enhances collaboration and integration with external tools through robust API support, streamlining workflows and facilitating automation.

See More Details

Operational Support (DevOps)

Development of support functions necessary for the operation of data analysis platforms, such as notifications and re-execution, from the engineer's perspective
See More Details

Operational Support (DevOps)

Utilizing trocco for all operational tasks streamlines processes significantly, eliminating the need for custom development of functions such as periodic execution and error notifications, which are required when using tools like Embulk. Trocco provides a comprehensive suite of necessary operational functionalities, including team-based management of operational permissions for transfer settings and connection information. Its features support regular, scheduled executions, as well as advanced notification systems through Slack or email, which can be triggered by error counts or execution times, and a mechanism to prevent duplicate job executions in closely scheduled scenarios. This cohesive approach enhances efficiency and reduces the complexity of data operations management.

See More Details

Web behavior log collection SDK

Collect website log events with a dedicated SDK and transfer them to your favorite DWH
See More Details

Web behavior log collection SDK

With a dedicated software development kit (SDK), this tool enables the collection of web logs and event data, which can then be seamlessly transferred to the data warehouse of choice. It's an effective way to gather web analytics data for further analysis and insights. For Example by simply embedding JavaScript tags on your website, you can effortlessly collect essential data for web analysis, including page views (PV), events, and email opening logs, mirroring the comprehensive data collection capabilities of Google Analytics in terms of raw logs, session, and channel definitions. Moreover, these logs can be seamlessly transferred to your preferred Data Warehouse (DWH), provided it is compatible with trocco services, ensuring that your web analytics data is easily accessible for in-depth analysis and decision-making.

See More Details

Security

Equipped with functions such as encryption and IP restriction to handle important data
See More Details

Security

Recognizing the importance of securing sensitive data, trocco ensures stringent security for data and sensitive information, employing encrypted communication for data in transit and AWS KMS for the encryption and isolated storage of sensitive details like connection passwords. It enhances data integrity and privacy by launching a dedicated server, in the form of a new container, for each data transfer, ensuring no data mix-up between different companies' accounts. Post-transfer, containers are destroyed, minimizing customer data retention time. Furthermore, Trocco® meets rigorous security standards by offering AWS PrivateLink for secure, private internet-free communication with external services, catering to those with strict security requirements and seeking additional safeguarding for their data transfers.

See More Details

Support

A dedicated customer will be in charge and support the response and operation of failures.
See More Details

Support

trocco offers comprehensive support to ensure clients can effectively utilize our services, including post-implementation onboarding sessions, assistance with specific inquiries, and error analysis, all included within the advanced plan at no extra charges. This type support structure fosters a seamless user experience, and for more interactive and prompt communication, we provide support channels through Slack and email, available during weekday daytime hours, excluding public holidays and weekends. This approach ensures that users receive timely assistance and guidance, enabling them to maximize the benefits of trocco’s offerings.

See More Details
How it works

Automated data replication architecture

From source to destination, our core functionality automates the extract and load replication process for all of our connectors, so you can enjoy total pipeline peace of mind.
Data Integration/
Ingestion
Begin by swiftly connecting to any data source, enabling the collection of diverse datasets within minutes.
Data Transformation
Convert the ingested raw data into structured business data models that are ready for analysis.
Data Orchestration
Automate and optimize the entire data flow, from initial ingestion to final storage.

Still curious ?

Watch our live demo video to see the platform in action. Witness firsthand how our ETL pipelines can transform your data processes, making them more efficient and effective.
Book a Demo
Book a Demo

Frequently Asked Questions

01.
How to fix the error that occurs when the transfer volume from BigQuery is too large
Note This is a machine-translated version of the original Japanese article. Please understand that some of the information contained on this page may be inaccurate. summary When specifying BigQuery as the transfer source, an error may occur if ...
02.
How to specify elements and extract values when an array is included in the source column JSON
Note This is a machine-translated version of the original Japanese article. Please understand that some of the information contained on this page may be inaccurate. summary If the JSON of the transfer source column contains an array and you wan...
03.
How to generate a webhook URL in Slack
Note This is a machine-translated version of the original Japanese article. Please understand that some of the information contained on this page may be inaccurate. summary Learn how to issue the webhook URL required for notifications to Slack....
04.
Is it possible to increase the transfer speed?
Note This is a machine-translated version of the original Japanese article. Please understand that some of the information contained on this page may be inaccurate. summary When specifying BigQuery as the transfer source, an error may occur if ...
05.
Do you support transfers from an on-premise environment?
Note This is a machine-translated version of the original Japanese article. Please understand that some of the information contained on this page may be inaccurate. summary When specifying BigQuery as the transfer source, an error may occur if ...
06.
Do you support transfers from an on-premise environment?
Note This is a machine-translated version of the original Japanese article. Please understand that some of the information contained on this page may be inaccurate. summary When specifying BigQuery as the transfer source, an error may occur if ...

TROCCO is trusted partner and certified with several Hyper Scalers