andras/great-ai
andras/great-ai
Watch
1
0
Fork
You've already forked great-ai
 0
Code Issues Pull requests 2 Projects Releases 4 Packages 1 Wiki Activity Actions

Deployed 63a4598 with MkDocs version: 1.3.0

Browse source
This commit is contained in:
2022-07-09 21:12:02 +00:00
commit 38c201a13f
51 changed files with 7759 additions and 1330 deletions
Download patch file Download diff file Expand all files Collapse all files

180
404.html
View file

@ -12,7 +12,7 @@
<meta name="author" content="András Schmelczer">
<link rel="icon" href="/assets/images/favicon.png">
<link rel="icon" href="/media/favicon.ico">
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
@ -244,13 +244,74 @@
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_2" type="checkbox" id="__nav_2" >
<label class="md-nav__link" for="__nav_2">
Tutorial
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" aria-label="Tutorial" data-md-level="1">
<label class="md-nav__title" for="__nav_2">
<span class="md-nav__icon md-icon"></span>
Tutorial
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="/tutorials/" class="md-nav__link">
Tutorials
<a href="/tutorial/" class="md-nav__link">
Train and deploy a SOTA model
</a>
</li>
<li class="md-nav__item">
<a href="/tutorial/train/" class="md-nav__link">
Train your model
</a>
</li>
<li class="md-nav__item">
<a href="/tutorial/deploy/" class="md-nav__link">
Harden and deploy your app
</a>
</li>
</ul>
</nav>
</li>
@ -259,13 +320,107 @@
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3" type="checkbox" id="__nav_3" >
<label class="md-nav__link" for="__nav_3">
User Guides
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" aria-label="User Guides" data-md-level="1">
<label class="md-nav__title" for="__nav_3">
<span class="md-nav__icon md-icon"></span>
User Guides
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3_1" type="checkbox" id="__nav_3_1" >
<label class="md-nav__link" for="__nav_3_1">
Simple AI lifecycle with classification
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" aria-label="Simple AI lifecycle with classification" data-md-level="2">
<label class="md-nav__title" for="__nav_3_1">
<span class="md-nav__icon md-icon"></span>
Simple AI lifecycle with classification
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="/how-to-guides/" class="md-nav__link">
How-To Guides
<a href="/simple/data/" class="md-nav__link">
Train a domain classifier on the semantic scholar dataset
</a>
</li>
<li class="md-nav__item">
<a href="/simple/train/" class="md-nav__link">
Train a domain classifier on the semantic scholar dataset
</a>
</li>
<li class="md-nav__item">
<a href="/simple/deploy/" class="md-nav__link">
Train a domain classifier on the semantic scholar dataset
</a>
</li>
</ul>
</nav>
</li>
</ul>
</nav>
</li>
@ -297,21 +452,6 @@
<li class="md-nav__item">
<a href="/simple-mag/train/" class="md-nav__link">
Train a field of study (domain) classification of sentences on the MAG dataset
</a>
</li>
</ul>
</nav>
</div>

5
example_secrets.ini
View file

@ -1,5 +0,0 @@
aws_region_name = your_region_like_eu-west-2
aws_access_key_id = YOUR_ACCESS_KEY_ID
aws_secret_access_key = YOUR_VERY_SECRET_ACCESS_KEY
large_files_bucket_name = create_a_bucket_and_put_its_name_here
aws_endpoint_url = this is optional, for backblaze, use this: https://s3.us-west-002.backblazeb2.com

7
explanation/explanation.md
View file

@ -0,0 +1,7 @@
# Explanation
A lot more details and discussion about the problem context and approaches of GreatAI can be found in my thesis.
<div style="display: flex; justify-content: space-evenly;" markdown>
[::fontawesome-solid-graduation-cap: Download](thesis/main.py){ .md-button .md-button--primary }
</div>

214
explanation/index.html
View file

@ -12,7 +12,7 @@
<meta name="author" content="András Schmelczer">
<link rel="icon" href="../assets/images/favicon.png">
<link rel="icon" href="../media/favicon.ico">
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
@ -69,6 +69,11 @@
<label class="md-overlay" for="__drawer"></label>
<div data-md-component="skip">
<a href="#explanation" class="md-skip">
Skip to content
</a>
</div>
<div data-md-component="announce">
@ -244,13 +249,74 @@
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_2" type="checkbox" id="__nav_2" >
<label class="md-nav__link" for="__nav_2">
Tutorial
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" aria-label="Tutorial" data-md-level="1">
<label class="md-nav__title" for="__nav_2">
<span class="md-nav__icon md-icon"></span>
Tutorial
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../tutorials/" class="md-nav__link">
Tutorials
<a href="../tutorial/" class="md-nav__link">
Train and deploy a SOTA model
</a>
</li>
<li class="md-nav__item">
<a href="../tutorial/train/" class="md-nav__link">
Train your model
</a>
</li>
<li class="md-nav__item">
<a href="../tutorial/deploy/" class="md-nav__link">
Harden and deploy your app
</a>
</li>
</ul>
</nav>
</li>
@ -259,13 +325,107 @@
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3" type="checkbox" id="__nav_3" >
<label class="md-nav__link" for="__nav_3">
User Guides
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" aria-label="User Guides" data-md-level="1">
<label class="md-nav__title" for="__nav_3">
<span class="md-nav__icon md-icon"></span>
User Guides
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3_1" type="checkbox" id="__nav_3_1" >
<label class="md-nav__link" for="__nav_3_1">
Simple AI lifecycle with classification
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" aria-label="Simple AI lifecycle with classification" data-md-level="2">
<label class="md-nav__title" for="__nav_3_1">
<span class="md-nav__icon md-icon"></span>
Simple AI lifecycle with classification
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../how-to-guides/" class="md-nav__link">
How-To Guides
<a href="../simple/data/" class="md-nav__link">
Train a domain classifier on the semantic scholar dataset
</a>
</li>
<li class="md-nav__item">
<a href="../simple/train/" class="md-nav__link">
Train a domain classifier on the semantic scholar dataset
</a>
</li>
<li class="md-nav__item">
<a href="../simple/deploy/" class="md-nav__link">
Train a domain classifier on the semantic scholar dataset
</a>
</li>
</ul>
</nav>
</li>
</ul>
</nav>
</li>
@ -296,6 +456,8 @@
<input class="md-nav__toggle md-toggle" data-md-toggle="toc" type="checkbox" id="__toc">
<a href="./" class="md-nav__link md-nav__link--active">
Explanation
@ -305,21 +467,6 @@
<li class="md-nav__item">
<a href="../simple-mag/train/" class="md-nav__link">
Train a field of study (domain) classification of sentences on the MAG dataset
</a>
</li>
</ul>
</nav>
</div>
@ -337,6 +484,8 @@
</nav>
</div>
@ -357,16 +506,18 @@
<h1>Explanation</h1>
<h1 id="explanation">Explanation<a class="headerlink" href="#explanation" title="Permanent link">#</a></h1>
<p>A lot more details and discussion about the problem context and approaches of GreatAI can be found in my thesis.</p>
<div style="display: flex; justify-content: space-evenly;">
<p><a class="md-button md-button--primary" href="thesis/main.py">:<span class="twemoji"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 640 512"><!--! Font Awesome Free 6.1.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2022 Fonticons, Inc.--><path d="M623.1 136.9 340.4 35.7a62.951 62.951 0 0 0-42.43 0L16.05 136.9C6.438 140.4 0 149.6 0 160s6.438 19.65 16.05 23.09l60.02 21.51c-11.89 15.8-20.26 34.16-24.55 53.95C40.05 263.4 32 274.8 32 288c0 9.953 4.814 18.49 11.94 24.36l-24.83 149C17.48 471.1 25 480 34.89 480h58.22c9.887 0 17.41-8.879 15.78-18.63l-24.83-149C91.19 306.5 96 297.1 96 288c0-10.29-5.174-19.03-12.72-24.89a106.301 106.301 0 0 1 24.94-47.03l190.6 68.23a62.951 62.951 0 0 0 42.43 0l282.7-101.2C633.6 179.6 640 170.4 640 160s-6.4-19.6-16.9-23.1zm-272 177.5c-9.4 3.7-20.2 5.6-31.1 5.6a94.767 94.767 0 0 1-32-5.555L142.8 262.5 128 405.3c0 41.3 85.1 74.7 192 74.7 105.1 0 192-33.4 192-74.67l-14.78-142.9L351.1 314.4z"/></svg></span> Download</a></p>
</div>
<hr>
<div class="md-source-file">
<small>
Last update:
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 8, 2022</span>
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 9, 2022</span>
</small>
@ -402,21 +553,6 @@
</a>
<a href="../simple-mag/train/" class="md-footer__link md-footer__link--next" aria-label="Next: Train a field of study (domain) classification of sentences on the MAG dataset" rel="next">
<div class="md-footer__title">
<div class="md-ellipsis">
<span class="md-footer__direction">
Next
</span>
Train a field of study (domain) classification of sentences on the MAG dataset
</div>
</div>
<div class="md-footer__button md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M4 11v2h12l-5.5 5.5 1.42 1.42L19.84 12l-7.92-7.92L10.5 5.5 16 11H4Z"/></svg>
</div>
</a>
</nav>
<div class="md-footer-meta md-typeset">

11
great_ai_example-main/Dockerfile
View file

@ -1,11 +0,0 @@
FROM test
COPY requirements.txt .
RUN pip install --no-cache-dir ./great_ai
COPY mongo.ini .
COPY deploy.ipynb .
# RUN python3 -m large_file --backend s3 -secrets ~/.aws/credentials --cache my_first_file.json:3 my_second_file my_folder:0
CMD ["deploy.ipynb"]

23
great_ai_example-main/README.md
View file

@ -1,23 +0,0 @@
# Train a domain classifier on the [semantic scholar dataset](https://api.semanticscholar.org/corpus)
![steps](diagrams/scope.svg)
## Install
### System dependencies
Make sure you have `python3`, `pip`, and `venv` installed.
> On Ubuntu, execute: `sudo apt install -y python3 python3-pip python3-venv`
### Install dependencies
```sh
python3 -m venv --copies .env
source .env/bin/activate
pip install -r requirements.txt
```
## Execute
- [Part 1](src/data.ipynb)
- [Part 2](src/train.ipynb)
- [Part 3](src/deploy.ipynb)

31
great_ai_example-main/format.sh
View file

@ -1,31 +0,0 @@
#!/bin/sh
set -e
echo "Installing dependencies if necessary"
python3 -m pip install --upgrade autoflake isort black[jupyter] mypy flake8
echo "Formatting and checking $1"
cd $1
echo Running autoflake
python3 -m autoflake --expand-star-imports --remove-all-unused-imports --ignore-init-module-imports --remove-unused-variables --in-place -r .
echo Running isort
python3 -m isort --profile black --skip .env .
echo Running black
python3 -m black . --exclude .env
if ls *.py 1> /dev/null 2>&1; then
echo Running mypy
python3 -m mypy --namespace-packages --ignore-missing-imports --install-types --non-interactive --disallow-untyped-defs --disallow-incomplete-defs --pretty --follow-imports=silent --exclude=external/ --exclude=/build/ .
fi
echo Running Flake8
python3 -m flake8 . --count --show-source --statistics --exclude=__init__.py,.env,external --ignore=E501,E722,E402,W503,E203
cd -
echo "Finished formatting"

471
great_ai_example-main/index.html
View file

@ -1,471 +0,0 @@
<!doctype html>
<html lang="en" class="no-js">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<meta name="description" content="GreatAI helps you easily transform your prototype AI code into production-ready software.">
<meta name="author" content="András Schmelczer">
<link rel="icon" href="../assets/images/favicon.png">
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
<title>Train a domain classifier on the [semantic scholar dataset](https://api.semanticscholar.org/corpus) - GreatAI documentation</title>
<link rel="stylesheet" href="../assets/stylesheets/main.1d29e8d0.min.css">
<link rel="stylesheet" href="../assets/stylesheets/palette.cbb835fc.min.css">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Roboto:300,300i,400,400i,700,700i%7CRoboto+Mono:400,400i,700,700i&display=fallback">
<style>:root{--md-text-font:"Roboto";--md-code-font:"Roboto Mono"}</style>
<link rel="stylesheet" href="../assets/_mkdocstrings.css">
<script>__md_scope=new URL("..",location),__md_get=(e,_=localStorage,t=__md_scope)=>JSON.parse(_.getItem(t.pathname+"."+e)),__md_set=(e,_,t=localStorage,a=__md_scope)=>{try{t.setItem(a.pathname+"."+e,JSON.stringify(_))}catch(e){}}</script>
</head>
<body dir="ltr" data-md-color-scheme="default" data-md-color-primary="" data-md-color-accent="">
<script>var palette=__md_get("__palette");if(palette&&"object"==typeof palette.color)for(var key of Object.keys(palette.color))document.body.setAttribute("data-md-color-"+key,palette.color[key])</script>
<input class="md-toggle" data-md-toggle="drawer" type="checkbox" id="__drawer" autocomplete="off">
<input class="md-toggle" data-md-toggle="search" type="checkbox" id="__search" autocomplete="off">
<label class="md-overlay" for="__drawer"></label>
<div data-md-component="skip">
<a href="#train-a-domain-classifier-on-the-semantic-scholar-dataset" class="md-skip">
Skip to content
</a>
</div>
<div data-md-component="announce">
</div>
<header class="md-header" data-md-component="header">
<nav class="md-header__inner md-grid" aria-label="Header">
<a href=".." title="GreatAI documentation" class="md-header__button md-logo" aria-label="GreatAI documentation" data-md-component="logo">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54Z"/></svg>
</a>
<label class="md-header__button md-icon" for="__drawer">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M3 6h18v2H3V6m0 5h18v2H3v-2m0 5h18v2H3v-2Z"/></svg>
</label>
<div class="md-header__title" data-md-component="header-title">
<div class="md-header__ellipsis">
<div class="md-header__topic">
<span class="md-ellipsis">
GreatAI documentation
</span>
</div>
<div class="md-header__topic" data-md-component="header-topic">
<span class="md-ellipsis">
Train a domain classifier on the [semantic scholar dataset](https://api.semanticscholar.org/corpus)
</span>
</div>
</div>
</div>
<form class="md-header__option" data-md-component="palette">
<input class="md-option" data-md-color-media="" data-md-color-scheme="default" data-md-color-primary="" data-md-color-accent="" aria-label="Switch to dark mode" type="radio" name="__palette" id="__palette_1">
<label class="md-header__button md-icon" title="Switch to dark mode" for="__palette_2" hidden>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 8a4 4 0 0 0-4 4 4 4 0 0 0 4 4 4 4 0 0 0 4-4 4 4 0 0 0-4-4m0 10a6 6 0 0 1-6-6 6 6 0 0 1 6-6 6 6 0 0 1 6 6 6 6 0 0 1-6 6m8-9.31V4h-4.69L12 .69 8.69 4H4v4.69L.69 12 4 15.31V20h4.69L12 23.31 15.31 20H20v-4.69L23.31 12 20 8.69Z"/></svg>
</label>
<input class="md-option" data-md-color-media="" data-md-color-scheme="slate" data-md-color-primary="" data-md-color-accent="" aria-label="Switch to light mode" type="radio" name="__palette" id="__palette_2">
<label class="md-header__button md-icon" title="Switch to light mode" for="__palette_1" hidden>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 18c-.89 0-1.74-.2-2.5-.55C11.56 16.5 13 14.42 13 12c0-2.42-1.44-4.5-3.5-5.45C10.26 6.2 11.11 6 12 6a6 6 0 0 1 6 6 6 6 0 0 1-6 6m8-9.31V4h-4.69L12 .69 8.69 4H4v4.69L.69 12 4 15.31V20h4.69L12 23.31 15.31 20H20v-4.69L23.31 12 20 8.69Z"/></svg>
</label>
</form>
<label class="md-header__button md-icon" for="__search">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.516 6.516 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5Z"/></svg>
</label>
<div class="md-search" data-md-component="search" role="dialog">
<label class="md-search__overlay" for="__search"></label>
<div class="md-search__inner" role="search">
<form class="md-search__form" name="search">
<input type="text" class="md-search__input" name="query" aria-label="Search" placeholder="Search" autocapitalize="off" autocorrect="off" autocomplete="off" spellcheck="false" data-md-component="search-query" required>
<label class="md-search__icon md-icon" for="__search">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.516 6.516 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5Z"/></svg>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20 11v2H8l5.5 5.5-1.42 1.42L4.16 12l7.92-7.92L13.5 5.5 8 11h12Z"/></svg>
</label>
<nav class="md-search__options" aria-label="Search">
<button type="reset" class="md-search__icon md-icon" aria-label="Clear" tabindex="-1">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M19 6.41 17.59 5 12 10.59 6.41 5 5 6.41 10.59 12 5 17.59 6.41 19 12 13.41 17.59 19 19 17.59 13.41 12 19 6.41Z"/></svg>
</button>
</nav>
</form>
<div class="md-search__output">
<div class="md-search__scrollwrap" data-md-scrollfix>
<div class="md-search-result" data-md-component="search-result">
<div class="md-search-result__meta">
Initializing search
</div>
<ol class="md-search-result__list"></ol>
</div>
</div>
</div>
</div>
</div>
<div class="md-header__source">
<a href="https://github.com/schmelczer/great-ai" title="Go to repository" class="md-source" data-md-component="source">
<div class="md-source__icon md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 6.1.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2022 Fonticons, Inc.--><path d="M439.55 236.05 244 40.45a28.87 28.87 0 0 0-40.81 0l-40.66 40.63 51.52 51.52c27.06-9.14 52.68 16.77 43.39 43.68l49.66 49.66c34.23-11.8 61.18 31 35.47 56.69-26.49 26.49-70.21-2.87-56-37.34L240.22 199v121.85c25.3 12.54 22.26 41.85 9.08 55a34.34 34.34 0 0 1-48.55 0c-17.57-17.6-11.07-46.91 11.25-56v-123c-20.8-8.51-24.6-30.74-18.64-45L142.57 101 8.45 235.14a28.86 28.86 0 0 0 0 40.81l195.61 195.6a28.86 28.86 0 0 0 40.8 0l194.69-194.69a28.86 28.86 0 0 0 0-40.81z"/></svg>
</div>
<div class="md-source__repository">
schmelczer/great-ai
</div>
</a>
</div>
</nav>
</header>
<div class="md-container" data-md-component="container">
<main class="md-main" data-md-component="main">
<div class="md-main__inner md-grid">
<div class="md-sidebar md-sidebar--primary" data-md-component="sidebar" data-md-type="navigation" >
<div class="md-sidebar__scrollwrap">
<div class="md-sidebar__inner">
<nav class="md-nav md-nav--primary" aria-label="Navigation" data-md-level="0">
<label class="md-nav__title" for="__drawer">
<a href=".." title="GreatAI documentation" class="md-nav__button md-logo" aria-label="GreatAI documentation" data-md-component="logo">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54Z"/></svg>
</a>
GreatAI documentation
</label>
<div class="md-nav__source">
<a href="https://github.com/schmelczer/great-ai" title="Go to repository" class="md-source" data-md-component="source">
<div class="md-source__icon md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 6.1.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2022 Fonticons, Inc.--><path d="M439.55 236.05 244 40.45a28.87 28.87 0 0 0-40.81 0l-40.66 40.63 51.52 51.52c27.06-9.14 52.68 16.77 43.39 43.68l49.66 49.66c34.23-11.8 61.18 31 35.47 56.69-26.49 26.49-70.21-2.87-56-37.34L240.22 199v121.85c25.3 12.54 22.26 41.85 9.08 55a34.34 34.34 0 0 1-48.55 0c-17.57-17.6-11.07-46.91 11.25-56v-123c-20.8-8.51-24.6-30.74-18.64-45L142.57 101 8.45 235.14a28.86 28.86 0 0 0 0 40.81l195.61 195.6a28.86 28.86 0 0 0 40.8 0l194.69-194.69a28.86 28.86 0 0 0 0-40.81z"/></svg>
</div>
<div class="md-source__repository">
schmelczer/great-ai
</div>
</a>
</div>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href=".." class="md-nav__link">
Overview
</a>
</li>
<li class="md-nav__item">
<a href="../tutorials/" class="md-nav__link">
Tutorials
</a>
</li>
<li class="md-nav__item">
<a href="../how-to-guides/" class="md-nav__link">
How-To Guides
</a>
</li>
<li class="md-nav__item">
<a href="../reference/" class="md-nav__link">
Reference
</a>
</li>
<li class="md-nav__item">
<a href="../explanation/" class="md-nav__link">
Explanation
</a>
</li>
<li class="md-nav__item">
<a href="../simple-mag/train/" class="md-nav__link">
Train a field of study (domain) classification of sentences on the MAG dataset
</a>
</li>
</ul>
</nav>
</div>
</div>
</div>
<div class="md-sidebar md-sidebar--secondary" data-md-component="sidebar" data-md-type="toc" >
<div class="md-sidebar__scrollwrap">
<div class="md-sidebar__inner">
<nav class="md-nav md-nav--secondary" aria-label="Table of contents">
<label class="md-nav__title" for="__toc">
<span class="md-nav__icon md-icon"></span>
Table of contents
</label>
<ul class="md-nav__list" data-md-component="toc" data-md-scrollfix>
<li class="md-nav__item">
<a href="#install" class="md-nav__link">
Install
</a>
<nav class="md-nav" aria-label="Install">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#system-dependencies" class="md-nav__link">
System dependencies
</a>
</li>
<li class="md-nav__item">
<a href="#install-dependencies" class="md-nav__link">
Install dependencies
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="#execute" class="md-nav__link">
Execute
</a>
</li>
</ul>
</nav>
</div>
</div>
</div>
<div class="md-content" data-md-component="content">
<article class="md-content__inner md-typeset">
<a href="https://github.com/schmelczer/great-ai/edit/main/docs/great_ai_example-main/README.md" title="Edit this page" class="md-content__button md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20.71 7.04c.39-.39.39-1.04 0-1.41l-2.34-2.34c-.37-.39-1.02-.39-1.41 0l-1.84 1.83 3.75 3.75M3 17.25V21h3.75L17.81 9.93l-3.75-3.75L3 17.25Z"/></svg>
</a>
<h1 id="train-a-domain-classifier-on-the-semantic-scholar-dataset">Train a domain classifier on the <a href="https://api.semanticscholar.org/corpus">semantic scholar dataset</a><a class="headerlink" href="#train-a-domain-classifier-on-the-semantic-scholar-dataset" title="Permanent link">#</a></h1>
<p><img alt="steps" src="diagrams/scope.svg" /></p>
<h2 id="install">Install<a class="headerlink" href="#install" title="Permanent link">#</a></h2>
<h3 id="system-dependencies">System dependencies<a class="headerlink" href="#system-dependencies" title="Permanent link">#</a></h3>
<p>Make sure you have <code>python3</code>, <code>pip</code>, and <code>venv</code> installed.</p>
<blockquote>
<p>On Ubuntu, execute: <code>sudo apt install -y python3 python3-pip python3-venv</code></p>
</blockquote>
<h3 id="install-dependencies">Install dependencies<a class="headerlink" href="#install-dependencies" title="Permanent link">#</a></h3>
<div class="highlight"><pre><span></span><code><a id="__codelineno-0-1" name="__codelineno-0-1" href="#__codelineno-0-1"></a>python3 -m venv --copies .env
<a id="__codelineno-0-2" name="__codelineno-0-2" href="#__codelineno-0-2"></a><span class="nb">source</span> .env/bin/activate
<a id="__codelineno-0-3" name="__codelineno-0-3" href="#__codelineno-0-3"></a>
<a id="__codelineno-0-4" name="__codelineno-0-4" href="#__codelineno-0-4"></a>pip install -r requirements.txt
</code></pre></div>
<h2 id="execute">Execute<a class="headerlink" href="#execute" title="Permanent link">#</a></h2>
<ul>
<li><a href="src/data/">Part 1</a></li>
<li><a href="src/train/">Part 2</a></li>
<li><a href="src/deploy/">Part 3</a></li>
</ul>
<hr>
<div class="md-source-file">
<small>
Last update:
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 8, 2022</span>
</small>
</div>
</article>
</div>
</div>
</main>
<footer class="md-footer">
<div class="md-footer-meta md-typeset">
<div class="md-footer-meta__inner md-grid">
<div class="md-copyright">
Made with
<a href="https://squidfunk.github.io/mkdocs-material/" target="_blank" rel="noopener">
Material for MkDocs
</a>
</div>
</div>
</div>
</footer>
</div>
<div class="md-dialog" data-md-component="dialog">
<div class="md-dialog__inner md-typeset"></div>
</div>
<script id="__config" type="application/json">{"base": "..", "features": ["content.code.annotate", "content.tooltips"], "search": "../assets/javascripts/workers/search.b97dbffb.min.js", "translations": {"clipboard.copied": "Copied to clipboard", "clipboard.copy": "Copy to clipboard", "search.config.lang": "en", "search.config.pipeline": "trimmer, stopWordFilter", "search.config.separator": "[\\s\\-]+", "search.placeholder": "Search", "search.result.more.one": "1 more on this page", "search.result.more.other": "# more on this page", "search.result.none": "No matching documents", "search.result.one": "1 matching document", "search.result.other": "# matching documents", "search.result.placeholder": "Type to start searching", "search.result.term.missing": "Missing", "select.version.title": "Select version"}}</script>
<script src="../assets/javascripts/bundle.6c7ad80a.min.js"></script>
</body>
</html>

4
great_ai_example-main/requirements.txt
View file

@ -1,4 +0,0 @@
great_ai
notebook
nbformat
nbconvert

2
great_ai_example-main/src/mongo.ini
View file

@ -1,2 +0,0 @@
mongo_connection_string=mongodb://localhost:27017/ # change this
mongo_database=great_ai_example # this will be automatically created

87
hello_world/hello_world.ipynb
View file

@ -1,87 +0,0 @@
{
"cells": [
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[38;5;226m2022-07-08 10:26:12,806 | WARNING | Environment variable ENVIRONMENT is not set, defaulting to development mode ‼️\u001b[0m\n",
"\u001b[38;5;226m2022-07-08 10:26:12,807 | WARNING | Cannot find credentials files, defaulting to using ParallelTinyDbDriver\u001b[0m\n",
"\u001b[38;5;226m2022-07-08 10:26:12,807 | WARNING | The selected tracing database (ParallelTinyDbDriver) is not recommended for production\u001b[0m\n",
"\u001b[38;5;226m2022-07-08 10:26:12,808 | WARNING | Cannot find credentials files, defaulting to using LargeFileLocal\u001b[0m\n",
"\u001b[38;5;39m2022-07-08 10:26:12,808 | INFO | Settings: configured ✅\u001b[0m\n",
"\u001b[38;5;39m2022-07-08 10:26:12,809 | INFO | 🔩 tracing_database: ParallelTinyDbDriver\u001b[0m\n",
"\u001b[38;5;39m2022-07-08 10:26:12,809 | INFO | 🔩 large_file_implementation: LargeFileLocal\u001b[0m\n",
"\u001b[38;5;39m2022-07-08 10:26:12,811 | INFO | 🔩 is_production: False\u001b[0m\n",
"\u001b[38;5;39m2022-07-08 10:26:12,811 | INFO | 🔩 should_log_exception_stack: True\u001b[0m\n",
"\u001b[38;5;39m2022-07-08 10:26:12,812 | INFO | 🔩 prediction_cache_size: 512\u001b[0m\n",
"\u001b[38;5;39m2022-07-08 10:26:12,813 | INFO | 🔩 dashboard_table_size: 20\u001b[0m\n",
"\u001b[38;5;226m2022-07-08 10:26:12,813 | WARNING | You still need to check whether you follow all best practices before trusting your deployment.\u001b[0m\n",
"\u001b[38;5;226m2022-07-08 10:26:12,814 | WARNING | > Find out more at https://se-ml.github.io/practices/\u001b[0m\n"
]
}
],
"source": [
"from great_ai import GreatAI\n",
"from asyncio import sleep\n",
"\n",
"\n",
"@GreatAI.create\n",
"async def hello_world(name: str) -> str:\n",
" await sleep(1)\n",
" return f\"Hello {name}!\""
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"Trace(trace_id='27c5e581-ad6e-4d50-bb77-862dfb7963fb', created='2022-07-08T08:26:12.911975', original_execution_time_ms=1.933, logged_values={'arg:name:value': 'hi', 'arg:name:length': 2}, models=[], exception=None, output='Hello hi!', feedback=None, tags=['hello_world', 'online', 'development'])"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"await hello_world('hi')"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3.10.4 ('.env': venv)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.4"
},
"orig_nbformat": 4,
"vscode": {
"interpreter": {
"hash": "02dd6d3afbfa9fbbe1037d64ad9014965528a1ccad21929d6e72f466389a68ad"
}
}
},
"nbformat": 4,
"nbformat_minor": 2
}

8
hello_world/hello_world.py
View file

@ -1,8 +0,0 @@
from great_ai import GreatAI
@GreatAI.create
def hello_world(name: str) -> str:
"""Learn more about GreatAI at https://github.com/schmelczer/great-ai"""
return f"Hello {name}!"

224
how-to-guides/index.html
View file

@ -12,12 +12,12 @@
<meta name="author" content="András Schmelczer">
<link rel="icon" href="../assets/images/favicon.png">
<link rel="icon" href="../media/favicon.ico">
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
<title>How-To Guides - GreatAI documentation</title>
<title>Index - GreatAI documentation</title>
@ -98,7 +98,7 @@
<div class="md-header__topic" data-md-component="header-topic">
<span class="md-ellipsis">
How-To Guides
Index
</span>
</div>
@ -244,13 +244,74 @@
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_2" type="checkbox" id="__nav_2" >
<label class="md-nav__link" for="__nav_2">
Tutorial
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" aria-label="Tutorial" data-md-level="1">
<label class="md-nav__title" for="__nav_2">
<span class="md-nav__icon md-icon"></span>
Tutorial
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../tutorials/" class="md-nav__link">
Tutorials
<a href="../tutorial/" class="md-nav__link">
Train and deploy a SOTA model
</a>
</li>
<li class="md-nav__item">
<a href="../tutorial/train/" class="md-nav__link">
Train your model
</a>
</li>
<li class="md-nav__item">
<a href="../tutorial/deploy/" class="md-nav__link">
Harden and deploy your app
</a>
</li>
</ul>
</nav>
</li>
@ -258,19 +319,105 @@
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3" type="checkbox" id="__nav_3" >
<label class="md-nav__link" for="__nav_3">
User Guides
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" aria-label="User Guides" data-md-level="1">
<label class="md-nav__title" for="__nav_3">
<span class="md-nav__icon md-icon"></span>
User Guides
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item md-nav__item--active">
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle" data-md-toggle="toc" type="checkbox" id="__toc">
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3_1" type="checkbox" id="__nav_3_1" >
<a href="./" class="md-nav__link md-nav__link--active">
How-To Guides
<label class="md-nav__link" for="__nav_3_1">
Simple AI lifecycle with classification
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" aria-label="Simple AI lifecycle with classification" data-md-level="2">
<label class="md-nav__title" for="__nav_3_1">
<span class="md-nav__icon md-icon"></span>
Simple AI lifecycle with classification
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../simple/data/" class="md-nav__link">
Train a domain classifier on the semantic scholar dataset
</a>
</li>
<li class="md-nav__item">
<a href="../simple/train/" class="md-nav__link">
Train a domain classifier on the semantic scholar dataset
</a>
</li>
<li class="md-nav__item">
<a href="../simple/deploy/" class="md-nav__link">
Train a domain classifier on the semantic scholar dataset
</a>
</li>
</ul>
</nav>
</li>
</ul>
</nav>
</li>
@ -305,21 +452,6 @@
<li class="md-nav__item">
<a href="../simple-mag/train/" class="md-nav__link">
Train a field of study (domain) classification of sentences on the MAG dataset
</a>
</li>
</ul>
</nav>
</div>
@ -351,13 +483,13 @@
<a href="https://github.com/schmelczer/great-ai/edit/main/docs/how-to-guides.md" title="Edit this page" class="md-content__button md-icon">
<a href="https://github.com/schmelczer/great-ai/edit/main/docs/how-to-guides/index.md" title="Edit this page" class="md-content__button md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20.71 7.04c.39-.39.39-1.04 0-1.41l-2.34-2.34c-.37-.39-1.02-.39-1.41 0l-1.84 1.83 3.75 3.75M3 17.25V21h3.75L17.81 9.93l-3.75-3.75L3 17.25Z"/></svg>
</a>
<h1>How-To Guides</h1>
<h1>Index</h1>
@ -366,7 +498,7 @@
<small>
Last update:
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 8, 2022</span>
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 9, 2022</span>
</small>
@ -383,42 +515,6 @@
<footer class="md-footer">
<nav class="md-footer__inner md-grid" aria-label="Footer" >
<a href="../tutorials/" class="md-footer__link md-footer__link--prev" aria-label="Previous: Tutorials" rel="prev">
<div class="md-footer__button md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20 11v2H8l5.5 5.5-1.42 1.42L4.16 12l7.92-7.92L13.5 5.5 8 11h12Z"/></svg>
</div>
<div class="md-footer__title">
<div class="md-ellipsis">
<span class="md-footer__direction">
Previous
</span>
Tutorials
</div>
</div>
</a>
<a href="../reference/" class="md-footer__link md-footer__link--next" aria-label="Next: Reference" rel="next">
<div class="md-footer__title">
<div class="md-ellipsis">
<span class="md-footer__direction">
Next
</span>
Reference
</div>
</div>
<div class="md-footer__button md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M4 11v2h12l-5.5 5.5 1.42 1.42L19.84 12l-7.92-7.92L10.5 5.5 16 11H4Z"/></svg>
</div>
</a>
</nav>
<div class="md-footer-meta md-typeset">
<div class="md-footer-meta__inner md-grid">
<div class="md-copyright">

0
how-to-guides/index.md Normal file
View file

1629
how-to-guides/scraping/index.html Normal file
View file

File diff suppressed because one or more lines are too long

29
how-to-guides/scraping/scraping.ipynb Normal file
View file

@ -0,0 +1,29 @@
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import httpx\n",
"\n",
"transport = httpx.AsyncHTTPTransport(retries=2)\n",
"limits = httpx.Limits(max_connections=None)\n",
"timeout = httpx.Timeout(connect=60.0, read=300, write=60, pool=None)\n",
"async with httpx.AsyncClient(\n",
" transport=transport, limits=limits, timeout=timeout\n",
") as client:\n",
" pass"
]
}
],
"metadata": {
"language_info": {
"name": "python"
},
"orig_nbformat": 4
},
"nbformat": 4,
"nbformat_minor": 2
}

230
ideas.drawio
View file

File diff suppressed because one or more lines are too long

246
index.html
View file

@ -12,7 +12,7 @@
<meta name="author" content="András Schmelczer">
<link rel="icon" href="assets/images/favicon.png">
<link rel="icon" href="media/favicon.ico">
<meta name="generator" content="mkdocs-1.3.0, mkdocs-material-8.3.9">
@ -70,7 +70,7 @@
<div data-md-component="skip">
<a href="#overview-of-greatai" class="md-skip">
<a href="#features" class="md-skip">
Skip to content
</a>
@ -241,8 +241,6 @@
<input class="md-nav__toggle md-toggle" data-md-toggle="toc" type="checkbox" id="__toc">
<label class="md-nav__link md-nav__link--active" for="__toc">
Overview
@ -259,8 +257,6 @@
<label class="md-nav__title" for="__toc">
<span class="md-nav__icon md-icon"></span>
@ -294,6 +290,13 @@
Why GreatAI?
</a>
</li>
<li class="md-nav__item">
<a href="#production-use" class="md-nav__link">
Production use
</a>
</li>
</ul>
@ -311,13 +314,74 @@
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_2" type="checkbox" id="__nav_2" >
<label class="md-nav__link" for="__nav_2">
Tutorial
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" aria-label="Tutorial" data-md-level="1">
<label class="md-nav__title" for="__nav_2">
<span class="md-nav__icon md-icon"></span>
Tutorial
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="tutorials/" class="md-nav__link">
Tutorials
<a href="tutorial/" class="md-nav__link">
Train and deploy a SOTA model
</a>
</li>
<li class="md-nav__item">
<a href="tutorial/train/" class="md-nav__link">
Train your model
</a>
</li>
<li class="md-nav__item">
<a href="tutorial/deploy/" class="md-nav__link">
Harden and deploy your app
</a>
</li>
</ul>
</nav>
</li>
@ -326,13 +390,107 @@
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3" type="checkbox" id="__nav_3" >
<label class="md-nav__link" for="__nav_3">
User Guides
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" aria-label="User Guides" data-md-level="1">
<label class="md-nav__title" for="__nav_3">
<span class="md-nav__icon md-icon"></span>
User Guides
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle" data-md-toggle="__nav_3_1" type="checkbox" id="__nav_3_1" >
<label class="md-nav__link" for="__nav_3_1">
Simple AI lifecycle with classification
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" aria-label="Simple AI lifecycle with classification" data-md-level="2">
<label class="md-nav__title" for="__nav_3_1">
<span class="md-nav__icon md-icon"></span>
Simple AI lifecycle with classification
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="how-to-guides/" class="md-nav__link">
How-To Guides
<a href="simple/data/" class="md-nav__link">
Train a domain classifier on the semantic scholar dataset
</a>
</li>
<li class="md-nav__item">
<a href="simple/train/" class="md-nav__link">
Train a domain classifier on the semantic scholar dataset
</a>
</li>
<li class="md-nav__item">
<a href="simple/deploy/" class="md-nav__link">
Train a domain classifier on the semantic scholar dataset
</a>
</li>
</ul>
</nav>
</li>
</ul>
</nav>
</li>
@ -364,21 +522,6 @@
<li class="md-nav__item">
<a href="simple-mag/train/" class="md-nav__link">
Train a field of study (domain) classification of sentences on the MAG dataset
</a>
</li>
</ul>
</nav>
</div>
@ -396,8 +539,6 @@
<label class="md-nav__title" for="__toc">
<span class="md-nav__icon md-icon"></span>
@ -431,6 +572,13 @@
Why GreatAI?
</a>
</li>
<li class="md-nav__item">
<a href="#production-use" class="md-nav__link">
Production use
</a>
</li>
</ul>
@ -454,7 +602,11 @@
<h1 id="overview-of-greatai">Overview of GreatAI<a class="headerlink" href="#overview-of-greatai" title="Permanent link">#</a></h1>
<div style="display: flex; justify-content: space-between; align-items: center;">
<h1 style="margin: 0">Overview of GreatAI</h1>
<img src="media/logo.png" width=80>
</div>
<p><a href="https://github.com/schmelczer/great-ai/actions/workflows/check.yml"><img alt="Test" src="https://github.com/schmelczer/great-ai/actions/workflows/test.yml/badge.svg" /></a>
<a href="https://sonar.schmelczer.com/dashboard?id=great-ai"><img alt="Quality Gate Status" src="https://sonar.scoutinscience.com/api/project_badges/measure?project=great-ai&amp;metric=alert_status" /></a>
<a href="https://github.com/schmelczer/great-ai/actions/workflows/publish.yaml"><img alt="Publish on PyPI" src="https://github.com/schmelczer/great-ai/actions/workflows/publish.yaml/badge.svg" /></a>
@ -464,15 +616,15 @@
<details class="quote">
<summary>Case studies</summary>
<p>"There is a need to consider and adapt well established SE practices which have been ignored or had a very narrow focus in ML literature."
&mdash; <a href="https://ieeexplore.ieee.org/abstract/document/9359253">John et al.</a></p>
<p>"Finally, we have found that existing tools to aid Machine Learning development do not address the specificities of different projects, and thus, are seldom adopted by teams." &mdash; <a href="https://link.springer.com/article/10.1007/s10664-021-09993-1">Haakman et al.</a></p>
<p>"Because a mature system might end up being (at most) 5% machine learning code and (at least) 95% glue code, it may be less costly to create a clean native solution rather than re-use a generic package." &mdash; <a href="https://www.researchgate.net/profile/Todd-Phillips/publication/319769912_Hidden_Technical_Debt_in_Machine_Learning_Systems/links/61e716d68d338833e37a7fd6/Hidden-Technical-Debt-in-Machine-Learning-Systems.pdf">Sculley et al.</a></p>
<p>"For example, practice 25 is very important for “Traceability", yet relatively weakly adopted. We expect that the results from this type of analysis can, in the future, provide useful guidance for practitioners in terms of aiding them to assess their rate of adoption for each practice and to create roadmaps for improving their processes. &mdash; <a href="https://dl.acm.org/doi/abs/10.1145/3382494.3410681?casa_token=uCFz0dtDR6gAAAAA:4_8OMJ-5njwopYkB1KSGAu9JfbNq4nfa8LRE0fj84ckjfo-GgtcYQivZTGxal3M4haoA8r_xwpw">Serban et al.</a></p>
&mdash; <a href="https://ieeexplore.ieee.org/abstract/document/9359253" target="_blank">John et al.</a></p>
<p>"Finally, we have found that existing tools to aid Machine Learning development do not address the specificities of different projects, and thus, are seldom adopted by teams." &mdash; <a href="https://link.springer.com/article/10.1007/s10664-021-09993-1" target="_blank">Haakman et al.</a></p>
<p>"Because a mature system might end up being (at most) 5% machine learning code and (at least) 95% glue code, it may be less costly to create a clean native solution rather than re-use a generic package." &mdash; <a href="https://www.researchgate.net/profile/Todd-Phillips/publication/319769912_Hidden_Technical_Debt_in_Machine_Learning_Systems/links/61e716d68d338833e37a7fd6/Hidden-Technical-Debt-in-Machine-Learning-Systems.pdf" target="_blank">Sculley et al.</a></p>
<p>"For example, practice 25 is very important for “Traceability", yet relatively weakly adopted. We expect that the results from this type of analysis can, in the future, provide useful guidance for practitioners in terms of aiding them to assess their rate of adoption for each practice and to create roadmaps for improving their processes. &mdash; <a href="https://dl.acm.org/doi/abs/10.1145/3382494.3410681?casa_token=uCFz0dtDR6gAAAAA:4_8OMJ-5njwopYkB1KSGAu9JfbNq4nfa8LRE0fj84ckjfo-GgtcYQivZTGxal3M4haoA8r_xwpw" target="_blank">Serban et al.</a></p>
</details>
<h2 id="features">Features<a class="headerlink" href="#features" title="Permanent link">#</a></h2>
<ul class="task-list">
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Save prediction traces of each prediction including arguments and model versions</li>
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Save feedback and merge it into a ground-truth database</li>
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Save feedback and merge it into a ground-truth database <em><img alt="➡" class="twemoji" src="https://twemoji.maxcdn.com/v/latest/svg/27a1.svg" title=":arrow_right:" /> quasi-shadow deployment</em></li>
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Version and store models and data on shared infrastructure <em>(MongoDB GridFS, S3-compatible storage, shared local-volume)</em></li>
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Automatically scaffolded custom REST API (and OpenAPI schema) for easy integration</li>
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Input validation</li>
@ -482,11 +634,11 @@
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Built-in parallelisation (with support for multiprocessing, async, and mixed modes) for batch processing</li>
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Well-tested utilities for common NLP tasks (cleaning, language-tagging, sentence-segmentation, etc.)</li>
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> A simple, unified configuration interface</li>
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Fully-typed API for IntelliSense support</li>
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Fully-typed API for <a href="https://github.com/microsoft/pylance-release" target="_blank">Pylance</a> and <a href="http://mypy-lang.org" target="_blank">MyPy</a> support</li>
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Auto-reload for development</li>
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Deployable Jupyter Notebooks</li>
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Docker support for deployment</li>
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled checked/><span class="task-list-indicator"></span></label> Dashboard for high-level overview and searching traces</li>
<li class="task-list-item"><label class="task-list-control"><input type="checkbox" disabled/><span class="task-list-indicator"></span></label> Shadow deployment</li>
</ul>
<h2 id="hello-world">Hello world<a class="headerlink" href="#hello-world" title="Permanent link">#</a></h2>
<div class="highlight"><pre><span></span><code><a id="__codelineno-0-1" name="__codelineno-0-1" href="#__codelineno-0-1"></a>pip install great-ai
@ -517,7 +669,7 @@
<div class="highlight"><span class="filename">terminal</span><pre><span></span><code><a id="__codelineno-2-1" name="__codelineno-2-1" href="#__codelineno-2-1"></a>great-ai hello-world.py
</code></pre></div>
<blockquote>
<p>Navigate to <a href="http://127.0.0.1:6060/">localhost:6060</a> in your browser.</p>
<p>Navigate to <a href="http://127.0.0.1:6060">localhost:6060</a> in your browser.</p>
</blockquote>
<div style="display: flex; justify-content: space-evenly;">
<p><img alt="" loading="lazy" src="media/hello-world-dashboard.png" /></p>
@ -525,11 +677,11 @@
</div>
<div class="admonition success">
<p class="admonition-title">Success</p>
<p>Your GreatAI service is ready for production use. Many of the <a href="https://se-ml.github.io/" target="_blank">SE4ML best-practices</a> are configured and implemented automatically. To have full control over your service and to understand what else you might need to do in your use case, continue reading this documentation.</p>
<p>Your GreatAI service is ready for production use. Many of the <a href="https://se-ml.github.io" target="_blank">SE4ML best-practices</a> are configured and implemented automatically. To have full control over your service and to understand what else you might need to do in your use case, continue reading this documentation.</p>
</div>
<h2 id="why-is-this-great">Why is this GREAT?<a class="headerlink" href="#why-is-this-great" title="Permanent link">#</a></h2>
<p><img alt="scope of GreatAI" src="scope-simple.drawio.svg" /></p>
<p>GreatAI fits between the prototype and deployment phase of your (or your organisation's) AI development lifecycle. This is highlighted with blue in the diagram. Here, a number of best practices can be automatically implemented concerning the following 5 aspects:</p>
<p><img alt="scope of GreatAI" src="media/scope-simple.drawio.svg" /></p>
<p>GreatAI fits between the prototype and deployment phase of your (or your organisation's) AI development lifecycle. This is highlighted with blue in the diagram. Here, a number of best practices can be automatically implemented aiming to achieve the following attributes:</p>
<ul>
<li><strong>G</strong>eneral: use any Python library without restriction</li>
<li><strong>R</strong>obust: have error-handling and well-tested utilities out-of-the-box </li>
@ -538,19 +690,23 @@
<li><strong>T</strong>rustworthy: deploy models that you and society can confidently trust</li>
</ul>
<h2 id="why-greatai">Why GreatAI?<a class="headerlink" href="#why-greatai" title="Permanent link">#</a></h2>
<p>There are other, existing solutions aiming to facilitate this phase. <a href="https://aws.amazon.com/sagemaker/" target="_blank">Amazon SageMaker</a> and <a href="https://www.seldon.io/solutions/open-source-projects/core" target="_blank">Seldon Core</a> provide the most comprehensive suite of features. If you have the opportunity use those, do that because they're great.</p>
<p>However, research indicates that professionals rarely use them. This may be due to their inherent setup and operating complexity. GreatAI is designed to be as simple to use as possible. Its clear, high-level API and sensible default configuration makes it extremely easy to start using. Despite its relative simplicity over Seldon Core, it still implements many <a href="https://se-ml.github.io/" target="_blank">best-practices</a>, and thus, can meaningfully improve your deployment without requiring prohibitively large effort.</p>
<p>There are other, existing solutions aiming to facilitate this phase. <a href="https://aws.amazon.com/sagemaker" target="_blank">Amazon SageMaker</a> and <a href="https://www.seldon.io/solutions/open-source-projects/core" target="_blank">Seldon Core</a> provide the most comprehensive suite of features. If you have the opportunity use those, do that because they're great.</p>
<p>However, research indicates that professionals rarely use them. This may be due to their inherent setup and operating complexity. GreatAI is designed to be as simple to use as possible. Its clear, high-level API and sensible default configuration makes it extremely easy to start using. Despite its relative simplicity over Seldon Core, it still implements many <a href="https://se-ml.github.io" target="_blank">best-practices</a>, and thus, can meaningfully improve your deployment without requiring prohibitively large effort.</p>
<div style="display: flex; justify-content: space-evenly;">
<p><a class="md-button md-button--primary" href="https://pypi.org/project/great-ai/"><span class="twemoji"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 6.1.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2022 Fonticons, Inc.--><path d="M439.8 200.5c-7.7-30.9-22.3-54.2-53.4-54.2h-40.1v47.4c0 36.8-31.2 67.8-66.8 67.8H172.7c-29.2 0-53.4 25-53.4 54.3v101.8c0 29 25.2 46 53.4 54.3 33.8 9.9 66.3 11.7 106.8 0 26.9-7.8 53.4-23.5 53.4-54.3v-40.7H226.2v-13.6h160.2c31.1 0 42.6-21.7 53.4-54.2 11.2-33.5 10.7-65.7 0-108.6zM286.2 404c11.1 0 20.1 9.1 20.1 20.3 0 11.3-9 20.4-20.1 20.4-11 0-20.1-9.2-20.1-20.4.1-11.3 9.1-20.3 20.1-20.3zM167.8 248.1h106.8c29.7 0 53.4-24.5 53.4-54.3V91.9c0-29-24.4-50.7-53.4-55.6-35.8-5.9-74.7-5.6-106.8.1-45.2 8-53.4 24.7-53.4 55.6v40.7h106.9v13.6h-147c-31.1 0-58.3 18.7-66.8 54.2-9.8 40.7-10.2 66.1 0 108.6 7.6 31.6 25.7 54.2 56.8 54.2H101v-48.8c0-35.3 30.5-66.4 66.8-66.4zm-6.7-142.6c-11.1 0-20.1-9.1-20.1-20.3.1-11.3 9-20.4 20.1-20.4 11 0 20.1 9.2 20.1 20.4s-9 20.3-20.1 20.3z"/></svg></span> Find it on PyPI</a></p>
<p><a class="md-button md-button--primary" href="https://pypi.org/project/great-ai"><span class="twemoji"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 6.1.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2022 Fonticons, Inc.--><path d="M439.8 200.5c-7.7-30.9-22.3-54.2-53.4-54.2h-40.1v47.4c0 36.8-31.2 67.8-66.8 67.8H172.7c-29.2 0-53.4 25-53.4 54.3v101.8c0 29 25.2 46 53.4 54.3 33.8 9.9 66.3 11.7 106.8 0 26.9-7.8 53.4-23.5 53.4-54.3v-40.7H226.2v-13.6h160.2c31.1 0 42.6-21.7 53.4-54.2 11.2-33.5 10.7-65.7 0-108.6zM286.2 404c11.1 0 20.1 9.1 20.1 20.3 0 11.3-9 20.4-20.1 20.4-11 0-20.1-9.2-20.1-20.4.1-11.3 9.1-20.3 20.1-20.3zM167.8 248.1h106.8c29.7 0 53.4-24.5 53.4-54.3V91.9c0-29-24.4-50.7-53.4-55.6-35.8-5.9-74.7-5.6-106.8.1-45.2 8-53.4 24.7-53.4 55.6v40.7h106.9v13.6h-147c-31.1 0-58.3 18.7-66.8 54.2-9.8 40.7-10.2 66.1 0 108.6 7.6 31.6 25.7 54.2 56.8 54.2H101v-48.8c0-35.3 30.5-66.4 66.8-66.4zm-6.7-142.6c-11.1 0-20.1-9.1-20.1-20.3.1-11.3 9-20.4 20.1-20.4 11 0 20.1 9.2 20.1 20.4s-9 20.3-20.1 20.3z"/></svg></span> Find it on PyPI</a></p>
<p><a class="md-button md-button--primary" href="https://hub.docker.com/repository/docker/schmelczera/great-ai"><span class="twemoji"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 640 512"><!--! Font Awesome Free 6.1.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2022 Fonticons, Inc.--><path d="M349.9 236.3h-66.1v-59.4h66.1v59.4zm0-204.3h-66.1v60.7h66.1V32zm78.2 144.8H362v59.4h66.1v-59.4zm-156.3-72.1h-66.1v60.1h66.1v-60.1zm78.1 0h-66.1v60.1h66.1v-60.1zm276.8 100c-14.4-9.7-47.6-13.2-73.1-8.4-3.3-24-16.7-44.9-41.1-63.7l-14-9.3-9.3 14c-18.4 27.8-23.4 73.6-3.7 103.8-8.7 4.7-25.8 11.1-48.4 10.7H2.4c-8.7 50.8 5.8 116.8 44 162.1 37.1 43.9 92.7 66.2 165.4 66.2 157.4 0 273.9-72.5 328.4-204.2 21.4.4 67.6.1 91.3-45.2 1.5-2.5 6.6-13.2 8.5-17.1l-13.3-8.9zm-511.1-27.9h-66v59.4h66.1v-59.4zm78.1 0h-66.1v59.4h66.1v-59.4zm78.1 0h-66.1v59.4h66.1v-59.4zm-78.1-72.1h-66.1v60.1h66.1v-60.1z"/></svg></span> Find it on DockerHub</a></p>
</div>
<h2 id="production-use">Production use<a class="headerlink" href="#production-use" title="Permanent link">#</a></h2>
<p>GreatAI has been battle-tested on the core platform services of <a href="https://www.scoutinscience.com/" target="_blank">ScoutinScience</a>.</p>
<p><img alt="ScoutinScience logo" src="media/scoutinscience.svg#only-light" />
<img alt="ScoutinScience logo" src="media/scoutinscience-white.svg#only-dark" /></p>
<hr>
<div class="md-source-file">
<small>
Last update:
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 8, 2022</span>
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 9, 2022</span>
</small>
@ -572,13 +728,13 @@
<a href="tutorials/" class="md-footer__link md-footer__link--next" aria-label="Next: Tutorials" rel="next">
<a href="tutorial/" class="md-footer__link md-footer__link--next" aria-label="Next: Train and deploy a SOTA model" rel="next">
<div class="md-footer__title">
<div class="md-ellipsis">
<span class="md-footer__direction">
Next
</span>
Tutorials
Train and deploy a SOTA model
</div>
</div>
<div class="md-footer__button md-icon">

45
index.md
View file

@ -1,29 +1,30 @@
# Overview of GreatAI
<div style="display: flex; justify-content: space-between; align-items: center;">
<h1 style="margin: 0">Overview of GreatAI</h1>
<img src="media/logo.png" width=80>
</div>
[![Test](https://github.com/schmelczer/great-ai/actions/workflows/test.yml/badge.svg)](https://github.com/schmelczer/great-ai/actions/workflows/check.yml)
[![Quality Gate Status](https://sonar.scoutinscience.com/api/project_badges/measure?project=great-ai&metric=alert_status)](https://sonar.schmelczer.com/dashboard?id=great-ai)
[![Publish on PyPI](https://github.com/schmelczer/great-ai/actions/workflows/publish.yaml/badge.svg)](https://github.com/schmelczer/great-ai/actions/workflows/publish.yaml)
[![Publish on DockerHub](https://github.com/schmelczer/great-ai/actions/workflows/docker.yaml/badge.svg)](https://github.com/schmelczer/great-ai/actions/workflows/docker.yaml)
[![Downloads](https://pepy.tech/badge/great-ai/month)](https://pepy.tech/project/great-ai)
Applying AI is becoming increasingly easier but many case studies have shown that these applications are often deployed poorly. This may lead to suboptimal performance and to introducing [unintended biases](https://en.wikipedia.org/wiki/Weapons_of_Math_Destruction){ target=_blank }. To extend the list of available solutions, ==GreatAI helps you easily transform your prototype AI code into production-ready software.==
??? quote "Case studies"
"There is a need to consider and adapt well established SE practices which have been ignored or had a very narrow focus in ML literature."
&mdash; [John et al.](https://ieeexplore.ieee.org/abstract/document/9359253)
&mdash; [John et al.](https://ieeexplore.ieee.org/abstract/document/9359253){ target=_blank }
"Finally, we have found that existing tools to aid Machine Learning development do not address the specificities of different projects, and thus, are seldom adopted by teams." &mdash; [Haakman et al.](https://link.springer.com/article/10.1007/s10664-021-09993-1)
"Because a mature system might end up being (at most) 5% machine learning code and (at least) 95% glue code, it may be less costly to create a clean native solution rather than re-use a generic package." &mdash; [Sculley et al.](https://www.researchgate.net/profile/Todd-Phillips/publication/319769912_Hidden_Technical_Debt_in_Machine_Learning_Systems/links/61e716d68d338833e37a7fd6/Hidden-Technical-Debt-in-Machine-Learning-Systems.pdf)
"For example, practice 25 is very important for “Traceability", yet relatively weakly adopted. We expect that the results from this type of analysis can, in the future, provide useful guidance for practitioners in terms of aiding them to assess their rate of adoption for each practice and to create roadmaps for improving their processes. &mdash; [Serban et al.](https://dl.acm.org/doi/abs/10.1145/3382494.3410681?casa_token=uCFz0dtDR6gAAAAA:4_8OMJ-5njwopYkB1KSGAu9JfbNq4nfa8LRE0fj84ckjfo-GgtcYQivZTGxal3M4haoA8r_xwpw)
"Finally, we have found that existing tools to aid Machine Learning development do not address the specificities of different projects, and thus, are seldom adopted by teams." &mdash; [Haakman et al.](https://link.springer.com/article/10.1007/s10664-021-09993-1){ target=_blank }
"Because a mature system might end up being (at most) 5% machine learning code and (at least) 95% glue code, it may be less costly to create a clean native solution rather than re-use a generic package." &mdash; [Sculley et al.](https://www.researchgate.net/profile/Todd-Phillips/publication/319769912_Hidden_Technical_Debt_in_Machine_Learning_Systems/links/61e716d68d338833e37a7fd6/Hidden-Technical-Debt-in-Machine-Learning-Systems.pdf){ target=_blank }
"For example, practice 25 is very important for “Traceability", yet relatively weakly adopted. We expect that the results from this type of analysis can, in the future, provide useful guidance for practitioners in terms of aiding them to assess their rate of adoption for each practice and to create roadmaps for improving their processes. &mdash; [Serban et al.](https://dl.acm.org/doi/abs/10.1145/3382494.3410681?casa_token=uCFz0dtDR6gAAAAA:4_8OMJ-5njwopYkB1KSGAu9JfbNq4nfa8LRE0fj84ckjfo-GgtcYQivZTGxal3M4haoA8r_xwpw){ target=_blank }
## Features
- [x] Save prediction traces of each prediction including arguments and model versions
- [x] Save feedback and merge it into a ground-truth database
- [x] Save feedback and merge it into a ground-truth database *:arrow_right: quasi-shadow deployment*
- [x] Version and store models and data on shared infrastructure *(MongoDB GridFS, S3-compatible storage, shared local-volume)*
- [x] Automatically scaffolded custom REST API (and OpenAPI schema) for easy integration
- [x] Input validation
@ -33,11 +34,11 @@ Applying AI is becoming increasingly easier but many case studies have shown tha
- [x] Built-in parallelisation (with support for multiprocessing, async, and mixed modes) for batch processing
- [x] Well-tested utilities for common NLP tasks (cleaning, language-tagging, sentence-segmentation, etc.)
- [x] A simple, unified configuration interface
- [x] Fully-typed API for IntelliSense support
- [x] Fully-typed API for [Pylance](https://github.com/microsoft/pylance-release){ target=_blank } and [MyPy](http://mypy-lang.org){ target=_blank } support
- [x] Auto-reload for development
- [x] Deployable Jupyter Notebooks
- [x] Docker support for deployment
- [x] Dashboard for high-level overview and searching traces
- [ ] Shadow deployment
## Hello world
@ -66,7 +67,7 @@ def hello_world(name: str) -> str: #(2)
```sh title="terminal"
great-ai hello-world.py
```
> Navigate to [localhost:6060](http://127.0.0.1:6060/) in your browser.
> Navigate to [localhost:6060](http://127.0.0.1:6060) in your browser.
<div style="display: flex; justify-content: space-evenly;" markdown>
![](media/hello-world-dashboard.png){ loading=lazy }
@ -75,14 +76,13 @@ great-ai hello-world.py
</div>
!!! success
Your GreatAI service is ready for production use. Many of the [SE4ML best-practices](https://se-ml.github.io/){ target=_blank } are configured and implemented automatically. To have full control over your service and to understand what else you might need to do in your use case, continue reading this documentation.
Your GreatAI service is ready for production use. Many of the [SE4ML best-practices](https://se-ml.github.io){ target=_blank } are configured and implemented automatically. To have full control over your service and to understand what else you might need to do in your use case, continue reading this documentation.
## Why is this GREAT?
![scope of GreatAI](scope-simple.drawio.svg)
![scope of GreatAI](media/scope-simple.drawio.svg)
GreatAI fits between the prototype and deployment phase of your (or your organisation's) AI development lifecycle. This is highlighted with blue in the diagram. Here, a number of best practices can be automatically implemented concerning the following 5 aspects:
GreatAI fits between the prototype and deployment phase of your (or your organisation's) AI development lifecycle. This is highlighted with blue in the diagram. Here, a number of best practices can be automatically implemented aiming to achieve the following attributes:
- **G**eneral: use any Python library without restriction
- **R**obust: have error-handling and well-tested utilities out-of-the-box
@ -92,13 +92,20 @@ GreatAI fits between the prototype and deployment phase of your (or your organis
## Why GreatAI?
There are other, existing solutions aiming to facilitate this phase. [Amazon SageMaker](https://aws.amazon.com/sagemaker/){ target=_blank } and [Seldon Core](https://www.seldon.io/solutions/open-source-projects/core){ target=_blank } provide the most comprehensive suite of features. If you have the opportunity use those, do that because they're great.
There are other, existing solutions aiming to facilitate this phase. [Amazon SageMaker](https://aws.amazon.com/sagemaker){ target=_blank } and [Seldon Core](https://www.seldon.io/solutions/open-source-projects/core){ target=_blank } provide the most comprehensive suite of features. If you have the opportunity use those, do that because they're great.
However, research indicates that professionals rarely use them. This may be due to their inherent setup and operating complexity. GreatAI is designed to be as simple to use as possible. Its clear, high-level API and sensible default configuration makes it extremely easy to start using. Despite its relative simplicity over Seldon Core, it still implements many [best-practices](https://se-ml.github.io/){ target=_blank }, and thus, can meaningfully improve your deployment without requiring prohibitively large effort.
However, research indicates that professionals rarely use them. This may be due to their inherent setup and operating complexity. GreatAI is designed to be as simple to use as possible. Its clear, high-level API and sensible default configuration makes it extremely easy to start using. Despite its relative simplicity over Seldon Core, it still implements many [best-practices](https://se-ml.github.io){ target=_blank }, and thus, can meaningfully improve your deployment without requiring prohibitively large effort.
<div style="display: flex; justify-content: space-evenly;" markdown>
[:fontawesome-brands-python: Find it on PyPI](https://pypi.org/project/great-ai/){ .md-button .md-button--primary }
[:fontawesome-brands-python: Find it on PyPI](https://pypi.org/project/great-ai){ .md-button .md-button--primary }
[:fontawesome-brands-docker: Find it on DockerHub](https://hub.docker.com/repository/docker/schmelczera/great-ai){ .md-button .md-button--primary }
</div>
## Production use
GreatAI has been battle-tested on the core platform services of [ScoutinScience](https://www.scoutinscience.com/){ target=_blank }.
![ScoutinScience logo](media/scoutinscience.svg#only-light)
![ScoutinScience logo](media/scoutinscience-white.svg#only-dark)

BIN
media/favicon.ico Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 15 KiB

Some files were not shown because too many files have changed in this diff Show more