Baseline#

This notebook will do a simple pipeline based on fastai with transfer learning concepts using resnet architecture. Typical ideas I want to implement here

  1. Get a working pipeline with the current dataset ( using Fastai)

  2. Do a submission on Kaggle

  3. Do some iterations using simple data augmentations

We will split the train dataset into 80-20 split. The idea is to have an actual measurement of model performance looking at the training data. However, trick for kaggle here is to train the model on entire training dataset before submitting results and reviewing score on the test.

Note

  • Big Images are not easy to fit on GPU.

  • Resizing big image to smaller sizes tends to loose certain features

  • Next ideas

    • Apply Digit Cleaner concept

    • Create Clean Dataset / Visualize and review accuracy of digit cleaner

    • Figure out way to do a bounding box

    • After that we can apply a few ideas

      • Apply resnet after digit cleaner ( current data had 0.93 error_rate after 1 epoch)

      • Do bounding box and break down dataset into individual digits. Then do clustering to 10 categories. Then use labels to calculate sum_digit

      • Split into individual digit images -> Resize -> Merge (permutations) into single image then train model on new dataset

Imports#

%load_ext autoreload
%autoreload 2
import torch
torch.cuda.empty_cache()
!nvidia-smi
Sat Mar 12 09:21:11 2022       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.103.01   Driver Version: 470.103.01   CUDA Version: 11.4     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla K80           Off  | 00000000:00:1E.0 Off |                    0 |
| N/A   32C    P8    30W / 149W |      3MiB / 11441MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+
import fastai
from fastai.vision.all import *
from fastai.vision.all import *
from fastai.vision.widgets import *
from aiking.data.external import * #We need to import this after fastai modules
import warnings
from matplotlib import cm

warnings.filterwarnings("ignore")
path = untar_data("kaggle_competitions::ultra-mnist"); path

(path/"train").ls()
(#28000) [Path('/Landmark2/pdo/aiking/data/ultra-mnist/train/ypuccwrtnt.jpeg'),Path('/Landmark2/pdo/aiking/data/ultra-mnist/train/jsutblirhq.jpeg'),Path('/Landmark2/pdo/aiking/data/ultra-mnist/train/nbekltglqy.jpeg'),Path('/Landmark2/pdo/aiking/data/ultra-mnist/train/ercjirbxrl.jpeg'),Path('/Landmark2/pdo/aiking/data/ultra-mnist/train/cmuvttuftw.jpeg'),Path('/Landmark2/pdo/aiking/data/ultra-mnist/train/hmqywwuolp.jpeg'),Path('/Landmark2/pdo/aiking/data/ultra-mnist/train/iwigdcizvr.jpeg'),Path('/Landmark2/pdo/aiking/data/ultra-mnist/train/nekpgbwcjl.jpeg'),Path('/Landmark2/pdo/aiking/data/ultra-mnist/train/jeuyqrvgrt.jpeg'),Path('/Landmark2/pdo/aiking/data/ultra-mnist/train/qdrqstkfop.jpeg')...]
df_train = pd.read_csv(path/'train_train.csv')
df_train['id'] = df_train['id'] + ".jpeg"
df_train
id digit_sum
0 oyihlzmbal.jpeg 4
1 qghmkvofim.jpeg 9
2 fbrfhdmlzf.jpeg 1
3 kopyiltoku.jpeg 5
4 aozdekkrks.jpeg 7
... ... ...
22395 acishewpws.jpeg 22
22396 nmgyqgimgu.jpeg 22
22397 zqgvuxngmf.jpeg 20
22398 qugvixgehi.jpeg 12
22399 yhqkvjsdqr.jpeg 1

22400 rows × 2 columns

dls = ImageDataLoaders.from_df(df_train, path, folder='train_black', valid_pct=0.2, fn_col=0, label_col=1, bs=16, item_tfms=Resize(299)); dls
<fastai.data.core.DataLoaders at 0x65ca2612fd0>
dls.show_batch()
../../_images/02_baseline_10_0.png
!nvidia-smi
Sat Mar 12 09:36:40 2022       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.103.01   Driver Version: 470.103.01   CUDA Version: 11.4     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla K80           Off  | 00000000:00:1E.0 Off |                    0 |
| N/A   47C    P0    69W / 149W |    557MiB / 11441MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      5666      C   ...da/envs/aiking/bin/python      554MiB |
+-----------------------------------------------------------------------------+
learn = cnn_learner(dls, resnet34, metrics=error_rate); learn
<fastai.learner.Learner at 0x65ca1c3acd0>
learn.fine_tune(4)
0.00% [0/1 00:00<00:00]
epoch train_loss valid_loss error_rate time

26.70% [299/1120 19:16<52:54 4.7228]
learn