Skip to content

Commit

Permalink
Merge pull request #70 from Sanketika-Obsrv/label_help_fixes
Browse files Browse the repository at this point in the history
#OBS-I331 - Fix the storage page help text and functionality
  • Loading branch information
HarishGangula authored Nov 7, 2024
2 parents c1bd07a + 0d1827d commit 0197864
Show file tree
Hide file tree
Showing 4 changed files with 213 additions and 437 deletions.
15 changes: 6 additions & 9 deletions web-console-v2/src/assets/help/processing.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -7,24 +7,21 @@ const ProcessingHelpText = () =>{
<section className="section" id="section1">
<h1 className="contentsHeader">Data Validation</h1>
<p className="contentBody">
<ul className="customList">
<p><strong>Data Validation:</strong> All data is automatically validated to ensure accuracy and compliance with the schema. This includes:</p>
<ul>
<ul>
<li>Data type checks</li>
<li>Enum validations</li>
<li>Range validations</li>
<li>Minimum and maximum value checks</li>
<li>Required field enforcement</li>
</ul>
<p>Any mismatches or missing required fields will cause the data to fail validation.</p>
<p><strong>Additional Fields:</strong> Select how to handle fields not defined in the schema:</p>
<ul>
</ul>
<p>Any mismatches or missing required fields will cause the data to fail validation.</p>
<p><strong>Additional Fields:</strong> Select how to handle fields not defined in the schema:</p>
<ul>
<li><strong>Fail for New Fields (No):</strong> Processing will fail if any fields outside the schema are detected.</li>
<li><strong>Ignore New Fields (Yes):</strong> Additional fields will be ignored, and processing will continue.</li>
</ul>
<p>Choose the option that best fits your data requirements.</p>

</ul>
<p>Choose the option that best fits your data requirements.</p>
</p>
</section>

Expand Down
45 changes: 16 additions & 29 deletions web-console-v2/src/assets/help/storage.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -4,41 +4,28 @@ const StorageHelpText = () =>{
return (
<div className="displayContent">
<h1 className="contentsHeader">Setup Guide for Dataset Storage</h1>
<section className="section" id="section1">
<div className="displayContent">
<h1 className="contentsHeader">Dataset Storage</h1>
<h3 className="contentsHeader">Dataset Type</h3>
<p className="contentBody">
At present, there are three types of datasets that can be created: a regular dataset and a master dataset. While there is no significant difference in the majority of workflows, there are some key differences as outlined below which would impact the UI flows:
</p>
<ul className="customList">
<li className="contentBody"><strong>Master Dataset:</strong> Used for capturing master data (e.g., user, location, catalog)</li>
<li className="contentBody"><strong>Events</strong></li>
<li className="contentBody"><strong>Transactional</strong></li>
<section className="section highlighted" id="section1">
<h1 className="contentsHeader">Select Storage Type</h1>
<div className="contentBody">
<p>Choose from multiple storage options that best meet your dataset needs:</p>
<ul>
<li><strong>Data Lakehouse (Hudi):</strong> Stores datasets cost-effectively, ideal for data science and analytical workloads, especially for transactional data. Offers scalable storage but may have slower query responses (not ideal for real-time APIs).</li>
<li><strong>Real-time Store (Druid):</strong> Designed for telemetry or append-only data, this real-time OLAP store supports sub-second response times for real-time queries and aggregate analysis.</li>
<li><strong>Cache Store (Redis):</strong> Enabled only for &quot;Master&quot; datasets, this option is optimal for data denormalization in real-time due to its speed.</li>
</ul>
</div>
</section>

<section className="section" id="section2">
<div className="displayContent">
<h3 className="contentsHeader">Dataset Store</h3>
<ul className="customList">
<li className="contentBody"><strong>Lakehouse</strong></li>
<li className="contentBody"><strong>Real-time Store</strong></li>
<h1 className="contentsHeader">Configure Storage Keys:</h1>
<div className="contentBody">
<p>Choose appropriate keys to enable indexing, data storage, and updates.</p>
<ul>
<li><strong>Primary Key:</strong> Required for Lakehouse and Cache Store. Select a unique identifier field to support record updates. Ensure this key uniquely identifies each record in the dataset.</li>
<li><strong>Timestamp Key:</strong> Required for OLAP Store (Druid). Select a timestamp for time-based indexing, either by choosing &quot;Event Arrival Time&quot; for the event&apos;s arrival time into the system, or any valid date-time field from the dataset.</li>
<li><strong>Partition Key:</strong> Required for Lakehouse, optional for OLAP Store, and not necessary for Cache Store. Use a field that logically segments data for efficient storage and query performance.</li>
</ul>
</div>

</section>

<section className="section" id="section3">
<div className="displayContent">
<h3 className="contentsHeader">Indexing Config</h3>
<ul className="customList">
<li className="contentBody"><strong>A timestamp key for time-series analysis:</strong> The storage data type of this field should be date, date-time, epoch, or any other date/time type; and this field must be marked as required. User options would be to consider data arrival time as a timestamp key or select a date from one of the date fields. This is not applicable for a master dataset.</li>
<li className="contentBody"><strong>A primary key for dedup/updates to the data:</strong> Can be any text/number field. Required only for File/Application/Stream connector types. A primary key can be a combination of more than one field.</li>
<li className="contentBody"><strong>A partition key for tenanting, efficient querying of data:</strong> Can be any text/number field. Partition key can be a combination of more than one field.</li>
</ul>
</div>
</section>
</div>
);
}
Expand Down
119 changes: 0 additions & 119 deletions web-console-v2/src/pages/StepsPages/Storage/Schema.ts

This file was deleted.

Loading

0 comments on commit 0197864

Please sign in to comment.