Deployed e8b07b4 with MkDocs version: 1.3.1
This commit is contained in:
parent
f2faa76f96
commit
f254b85515
47 changed files with 146 additions and 128 deletions
2
404.html
2
404.html
|
|
@ -13,7 +13,7 @@
|
||||||
|
|
||||||
|
|
||||||
<link rel="icon" href="/media/favicon.ico">
|
<link rel="icon" href="/media/favicon.ico">
|
||||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -13,7 +13,7 @@
|
||||||
|
|
||||||
|
|
||||||
<link rel="icon" href="../../../media/favicon.ico">
|
<link rel="icon" href="../../../media/favicon.ico">
|
||||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -13,7 +13,7 @@
|
||||||
|
|
||||||
|
|
||||||
<link rel="icon" href="../../../media/favicon.ico">
|
<link rel="icon" href="../../../media/favicon.ico">
|
||||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -13,7 +13,7 @@
|
||||||
|
|
||||||
|
|
||||||
<link rel="icon" href="../../../media/favicon.ico">
|
<link rel="icon" href="../../../media/favicon.ico">
|
||||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -13,7 +13,7 @@
|
||||||
|
|
||||||
|
|
||||||
<link rel="icon" href="../../media/favicon.ico">
|
<link rel="icon" href="../../media/favicon.ico">
|
||||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -13,7 +13,7 @@
|
||||||
|
|
||||||
|
|
||||||
<link rel="icon" href="../../../media/favicon.ico">
|
<link rel="icon" href="../../../media/favicon.ico">
|
||||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -13,7 +13,7 @@
|
||||||
|
|
||||||
|
|
||||||
<link rel="icon" href="../../../media/favicon.ico">
|
<link rel="icon" href="../../../media/favicon.ico">
|
||||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -6,7 +6,7 @@
|
||||||
"source": [
|
"source": [
|
||||||
"# Simple example: data engineering\n",
|
"# Simple example: data engineering\n",
|
||||||
"\n",
|
"\n",
|
||||||
"Here, we solve a problem similar to the tutorial's but with an explainable Naive Bayes classifier and more best-practices. In short, we train a domain classifier on the [semantic scholar dataset](https://api.semanticscholar.org/corpus) by taking full advantage of `great-ai`. Subsequently, we create a production-ready deployment.\n",
|
"Here, we solve a problem similar to the tutorial's but with an explainable Naive Bayes classifier and more best practices. In short, we train a domain classifier on the [semantic scholar dataset](https://api.semanticscholar.org/corpus) by taking full advantage of `great-ai`. Subsequently, we create a production-ready deployment.\n",
|
||||||
"\n",
|
"\n",
|
||||||
"\n",
|
"\n",
|
||||||
"> The blue boxes show the steps of a typical AI-development lifecycle implemented in this notebook.\n",
|
"> The blue boxes show the steps of a typical AI-development lifecycle implemented in this notebook.\n",
|
||||||
|
|
|
||||||
|
|
@ -13,7 +13,7 @@
|
||||||
|
|
||||||
|
|
||||||
<link rel="icon" href="../../../media/favicon.ico">
|
<link rel="icon" href="../../../media/favicon.ico">
|
||||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -1988,7 +1988,7 @@ Licensed under the Apache License, Version 2.0.
|
||||||
</div>
|
</div>
|
||||||
<div class="jp-InputArea jp-Cell-inputArea"><div class="jp-InputPrompt jp-InputArea-prompt">
|
<div class="jp-InputArea jp-Cell-inputArea"><div class="jp-InputPrompt jp-InputArea-prompt">
|
||||||
</div><div class="jp-RenderedHTMLCommon jp-RenderedMarkdown jp-MarkdownOutput " data-mime-type="text/markdown">
|
</div><div class="jp-RenderedHTMLCommon jp-RenderedMarkdown jp-MarkdownOutput " data-mime-type="text/markdown">
|
||||||
<h1 id="simple-example-data-engineering">Simple example: data engineering<a class="anchor-link" href="#simple-example-data-engineering">¶</a></h1><p>Here, we solve a problem similar to the tutorial's but with an explainable Naive Bayes classifier and more best-practices. In short, we train a domain classifier on the <a href="https://api.semanticscholar.org/corpus">semantic scholar dataset</a> by taking full advantage of <code>great-ai</code>. Subsequently, we create a production-ready deployment.</p>
|
<h1 id="simple-example-data-engineering">Simple example: data engineering<a class="anchor-link" href="#simple-example-data-engineering">¶</a></h1><p>Here, we solve a problem similar to the tutorial's but with an explainable Naive Bayes classifier and more best practices. In short, we train a domain classifier on the <a href="https://api.semanticscholar.org/corpus">semantic scholar dataset</a> by taking full advantage of <code>great-ai</code>. Subsequently, we create a production-ready deployment.</p>
|
||||||
<p><img src="/media/scope-data.svg" alt="position of this step in the lifecycle"></p>
|
<p><img src="/media/scope-data.svg" alt="position of this step in the lifecycle"></p>
|
||||||
<blockquote><p>The blue boxes show the steps of a typical AI-development lifecycle implemented in this notebook.</p>
|
<blockquote><p>The blue boxes show the steps of a typical AI-development lifecycle implemented in this notebook.</p>
|
||||||
</blockquote>
|
</blockquote>
|
||||||
|
|
@ -2416,7 +2416,7 @@ add_ground_truth(X, y, train_split_ratio=0.8, test_split_ratio=0.2)</div>
|
||||||
<small>
|
<small>
|
||||||
|
|
||||||
Last update:
|
Last update:
|
||||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 16, 2022</span>
|
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 24, 2022</span>
|
||||||
|
|
||||||
|
|
||||||
</small>
|
</small>
|
||||||
|
|
|
||||||
|
|
@ -13,7 +13,7 @@
|
||||||
|
|
||||||
|
|
||||||
<link rel="icon" href="../../../media/favicon.ico">
|
<link rel="icon" href="../../../media/favicon.ico">
|
||||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -13,7 +13,7 @@
|
||||||
|
|
||||||
|
|
||||||
<link rel="icon" href="../../../media/favicon.ico">
|
<link rel="icon" href="../../../media/favicon.ico">
|
||||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -13,7 +13,7 @@
|
||||||
|
|
||||||
|
|
||||||
<link rel="icon" href="../../../media/favicon.ico">
|
<link rel="icon" href="../../../media/favicon.ico">
|
||||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -13,7 +13,7 @@
|
||||||
|
|
||||||
|
|
||||||
<link rel="icon" href="../media/favicon.ico">
|
<link rel="icon" href="../media/favicon.ico">
|
||||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -13,7 +13,7 @@
|
||||||
|
|
||||||
|
|
||||||
<link rel="icon" href="../../media/favicon.ico">
|
<link rel="icon" href="../../media/favicon.ico">
|
||||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -13,7 +13,7 @@
|
||||||
|
|
||||||
|
|
||||||
<link rel="icon" href="../../media/favicon.ico">
|
<link rel="icon" href="../../media/favicon.ico">
|
||||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -13,7 +13,7 @@
|
||||||
|
|
||||||
|
|
||||||
<link rel="icon" href="../../media/favicon.ico">
|
<link rel="icon" href="../../media/favicon.ico">
|
||||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -7,7 +7,7 @@ In order to simplify your training data management, `great-ai` provide two compl
|
||||||
At the start of your experiments' first iteration, after you've gathered suitable samples for training, you can call [great_ai.add_ground_truth][]. This automatically stores a timestamp and also allows you to assign tags to the data. Using these attributes, [great_ai.query_ground_truth][] can be called to get a filtered view of the training data.
|
At the start of your experiments' first iteration, after you've gathered suitable samples for training, you can call [great_ai.add_ground_truth][]. This automatically stores a timestamp and also allows you to assign tags to the data. Using these attributes, [great_ai.query_ground_truth][] can be called to get a filtered view of the training data.
|
||||||
|
|
||||||
!!! important "Train-test-validation splits"
|
!!! important "Train-test-validation splits"
|
||||||
It is a best-practice to lock-away the test split of your data that is only used for the final quality assessment. This prevents you from accidentally training on it, or inadvertently tuning the model to have the highest accuracy metrics on the test split. This, of course, may lead to dubious results, hence, care must be taken to avoid it.
|
It is a best practice to lock-away the test split of your data that is only used for the final quality assessment. This prevents you from accidentally training on it, or inadvertently tuning the model to have the highest accuracy metrics on the test split. This, of course, may lead to dubious results, hence, care must be taken to avoid it.
|
||||||
|
|
||||||
With [great_ai.add_ground_truth][], there is an option to tag the samples with `train`, `test`, and `validation` randomly, following a predefined distribution. This happens as soon as they're written in the database. Later, these can be queried by providing the name of the appropriate tags.
|
With [great_ai.add_ground_truth][], there is an option to tag the samples with `train`, `test`, and `validation` randomly, following a predefined distribution. This happens as soon as they're written in the database. Later, these can be queried by providing the name of the appropriate tags.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -13,7 +13,7 @@
|
||||||
|
|
||||||
|
|
||||||
<link rel="icon" href="../../media/favicon.ico">
|
<link rel="icon" href="../../media/favicon.ico">
|
||||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -912,7 +912,7 @@
|
||||||
<p>At the start of your experiments' first iteration, after you've gathered suitable samples for training, you can call <a class="autorefs autorefs-internal" href="../../reference/#great_ai.add_ground_truth">great_ai.add_ground_truth</a>. This automatically stores a timestamp and also allows you to assign tags to the data. Using these attributes, <a class="autorefs autorefs-internal" href="../../reference/#great_ai.query_ground_truth">great_ai.query_ground_truth</a> can be called to get a filtered view of the training data.</p>
|
<p>At the start of your experiments' first iteration, after you've gathered suitable samples for training, you can call <a class="autorefs autorefs-internal" href="../../reference/#great_ai.add_ground_truth">great_ai.add_ground_truth</a>. This automatically stores a timestamp and also allows you to assign tags to the data. Using these attributes, <a class="autorefs autorefs-internal" href="../../reference/#great_ai.query_ground_truth">great_ai.query_ground_truth</a> can be called to get a filtered view of the training data.</p>
|
||||||
<div class="admonition important">
|
<div class="admonition important">
|
||||||
<p class="admonition-title">Train-test-validation splits</p>
|
<p class="admonition-title">Train-test-validation splits</p>
|
||||||
<p>It is a best-practice to lock-away the test split of your data that is only used for the final quality assessment. This prevents you from accidentally training on it, or inadvertently tuning the model to have the highest accuracy metrics on the test split. This, of course, may lead to dubious results, hence, care must be taken to avoid it.</p>
|
<p>It is a best practice to lock-away the test split of your data that is only used for the final quality assessment. This prevents you from accidentally training on it, or inadvertently tuning the model to have the highest accuracy metrics on the test split. This, of course, may lead to dubious results, hence, care must be taken to avoid it.</p>
|
||||||
<p>With <a class="autorefs autorefs-internal" href="../../reference/#great_ai.add_ground_truth">great_ai.add_ground_truth</a>, there is an option to tag the samples with <code>train</code>, <code>test</code>, and <code>validation</code> randomly, following a predefined distribution. This happens as soon as they're written in the database. Later, these can be queried by providing the name of the appropriate tags.</p>
|
<p>With <a class="autorefs autorefs-internal" href="../../reference/#great_ai.add_ground_truth">great_ai.add_ground_truth</a>, there is an option to tag the samples with <code>train</code>, <code>test</code>, and <code>validation</code> randomly, following a predefined distribution. This happens as soon as they're written in the database. Later, these can be queried by providing the name of the appropriate tags.</p>
|
||||||
</div>
|
</div>
|
||||||
<p>The nice thing about this is that the 'input-expected output' pairs are stored as traces. Thus, they behave exactly like regular prediction traces.</p>
|
<p>The nice thing about this is that the 'input-expected output' pairs are stored as traces. Thus, they behave exactly like regular prediction traces.</p>
|
||||||
|
|
@ -968,7 +968,7 @@
|
||||||
<small>
|
<small>
|
||||||
|
|
||||||
Last update:
|
Last update:
|
||||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 15, 2022</span>
|
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 24, 2022</span>
|
||||||
|
|
||||||
|
|
||||||
</small>
|
</small>
|
||||||
|
|
|
||||||
|
|
@ -13,7 +13,7 @@
|
||||||
|
|
||||||
|
|
||||||
<link rel="icon" href="../../media/favicon.ico">
|
<link rel="icon" href="../../media/favicon.ico">
|
||||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -13,7 +13,7 @@
|
||||||
|
|
||||||
|
|
||||||
<link rel="icon" href="../../media/favicon.ico">
|
<link rel="icon" href="../../media/favicon.ico">
|
||||||
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
|
<meta name="generator" content="mkdocs-1.3.1, mkdocs-material-8.3.9">
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -1080,7 +1080,7 @@
|
||||||
<p>Lastly, all version of the remote object can be deleted by calling <code>LargeFileS3("my-file").delete()</code>. It will still reside in your local cache afterwards; its deletion will happen next time your local cache has to be pruned.</p>
|
<p>Lastly, all version of the remote object can be deleted by calling <code>LargeFileS3("my-file").delete()</code>. It will still reside in your local cache afterwards; its deletion will happen next time your local cache has to be pruned.</p>
|
||||||
</details>
|
</details>
|
||||||
<h2 id="from-the-command-line">From the command-line<a class="headerlink" href="#from-the-command-line" title="Permanent link">#</a></h2>
|
<h2 id="from-the-command-line">From the command-line<a class="headerlink" href="#from-the-command-line" title="Permanent link">#</a></h2>
|
||||||
<p>The main reason for using the <code>large-file</code> or <code>python3 -m great_ai.large_file</code> commands is to upload or download models from the terminal. For example, when building a docker image, it is best-practice to cache the referred models.</p>
|
<p>The main reason for using the <code>large-file</code> or <code>python3 -m great_ai.large_file</code> commands is to upload or download models from the terminal. For example, when building a docker image, it is best practice to cache the referred models.</p>
|
||||||
<h3 id="setup">Setup<a class="headerlink" href="#setup" title="Permanent link">#</a></h3>
|
<h3 id="setup">Setup<a class="headerlink" href="#setup" title="Permanent link">#</a></h3>
|
||||||
<p>Create an .ini file (or use <em>~/.aws/credentials</em>). It may look like this:</p>
|
<p>Create an .ini file (or use <em>~/.aws/credentials</em>). It may look like this:</p>
|
||||||
<div class="highlight"><pre><span></span><code><a id="__codelineno-4-1" name="__codelineno-4-1" href="#__codelineno-4-1"></a><span class="na">aws_region_name</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s">your_region_like_eu-west-2</span><span class="w"></span>
|
<div class="highlight"><pre><span></span><code><a id="__codelineno-4-1" name="__codelineno-4-1" href="#__codelineno-4-1"></a><span class="na">aws_region_name</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s">your_region_like_eu-west-2</span><span class="w"></span>
|
||||||
|
|
@ -1117,7 +1117,7 @@
|
||||||
<small>
|
<small>
|
||||||
|
|
||||||
Last update:
|
Last update:
|
||||||
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 16, 2022</span>
|
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 24, 2022</span>
|
||||||
|
|
||||||
|
|
||||||
</small>
|
</small>
|
||||||
|
|
|
||||||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Reference in a new issue