Tip 4 AI

• Remove spike
  • 
    
    
  • Options
    • Change network
      • I.e. use 2 layer LSTM instead of 1 layer LSTM
    • Increase the batch size
    • Remove outliers
• number of steps (tensorflow)
  
  • num_steps = (len(traindf) / args['batch_size']) / args['learning_rate']
• queue capacity (tensorflow)
  • Queue_capacity = batch_size * 10
    
• checkpoint (tensorflow)
  • save_checkpoints_steps = max(100, params["train_steps"] // 10)
    
• tf.data.Dataset (tensorflow)

  • num_parallel_X, prefetch

  • dataset.apply(

  • Tf.contrib.data.shuffle_and_repeat

  • Tf.contrib.data.map_and_batch

• Mirrored strategy (tensorflow)

  • distribution = tf.contrib.distribute.MirroredStrategy()

  • tf.estimator.RunConfig(train_distribute=distribution)
    

• Multi-label classification (tensorflow)
  • Use the loss functions listed below for reducing computation
    • tf.nn.sampled_softmax_loss, tf.nn.nce_loss
• Visualizing the convolutions and pooling (tensorflow)

  • Code

    import tensorflow as tf print(tf.__version__) mnist = tf.keras.datasets.fashion_mnist (training_images, training_labels), (test_images, test_labels) = mnist.load_data() training_images=training_images.reshape(60000, 28, 28, 1) training_images=training_images / 255.0 test_images = test_images.reshape(10000, 28, 28, 1) test_images=test_images/255.0 model = tf.keras.models.Sequential([   tf.keras.layers.Conv2D(64, (3,3), activation='relu', input_shape=(28, 28, 1)),   tf.keras.layers.MaxPooling2D(2, 2),   tf.keras.layers.Conv2D(64, (3,3), activation='relu'),   tf.keras.layers.MaxPooling2D(2,2),   tf.keras.layers.Flatten(),   tf.keras.layers.Dense(128, activation='relu'),   tf.keras.layers.Dense(10, activation='softmax') ]) model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy']) model.summary() model.fit(training_images, training_labels, epochs=5) test_loss = model.evaluate(test_images, test_labels)    print(test_labels[:100])    import matplotlib.pyplot as plt f, axarr = plt.subplots(3,4) FIRST_IMAGE=1 SECOND_IMAGE=3 THIRD_IMAGE=4 CONVOLUTION_NUMBER = 1 from tensorflow.keras import models layer_outputs = [layer.output for layer in model.layers] activation_model = tf.keras.models.Model(inputs = model.input, outputs = layer_outputs) for x in range(0,4):   f1 = activation_model.predict(test_images[FIRST_IMAGE].reshape(1, 28, 28, 1))[x]   axarr[0,x].imshow(f1[0, : , :, CONVOLUTION_NUMBER], cmap='inferno')   axarr[0,x].grid(False)   f2 = activation_model.predict(test_images[SECOND_IMAGE].reshape(1, 28, 28, 1))[x]   axarr[1,x].imshow(f2[0, : , :, CONVOLUTION_NUMBER], cmap='inferno')   axarr[1,x].grid(False)   f3 = activation_model.predict(test_images[THIRD_IMAGE].reshape(1, 28, 28, 1))[x]   axarr[2,x].imshow(f3[0, : , :, CONVOLUTION_NUMBER], cmap='inferno')   axarr[2,x].grid(False)

• Early stopping (tensorflow)

  • Code

    import tensorflow as tf print(tf.__version__)   class myCallback(tf.keras.callbacks.Callback):   def on_epoch_end(self, epoch, logs={}):     if(logs.get('loss')<0.4):       self.model.stop_training = True     elif(logs.get('acc')>0.8):       self.model.stop_training = True   callbacks = myCallback() mnist = tf.keras.datasets.fashion_mnist (training_images, training_labels), (test_images, test_labels) = mnist.load_data() training_images=training_images/255.0 test_images=test_images/255.0 model = tf.keras.models.Sequential([   tf.keras.layers.Flatten(),   tf.keras.layers.Dense(512, activation=tf.nn.relu),   tf.keras.layers.Dense(10, activation=tf.nn.softmax) ]) model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy']) model.fit(training_images, training_labels, epochs=5, callbacks=[callbacks])

LCS

Longest common subsequence

pystrgrp: https://drive.google.com/open?id=1Ig_ATnmLUJIuHbFPRGlvZdM3Xd5Yp32U

Example

from pystrgrp import Strgrp

def pystrgrp(strings):
    clusters = Strgrp(0.7)
    for string in (x.strip() for x in strings):
        seq, id = string.split(',')
        clusters.add(seq, id)
    return clusters

data = sorted(['12345,1','1234567,2','1234568,3','2345678,4',
               '2345679,5','345678,6','1234578,7','3456789,8','abcdefg,9','bcdefg,10'], reverse=0)

grps = pystrgrp(data)
grps

grps_list = [g for g in grps]
grps_list

import pandas as pd

df = pd.DataFrame()

for i in range(len(grps_list)):
    grp = [g for g in grps_list[i]]
 
    for j in range(len(grp)):
        print(i, grp[j].key(), grp[j].value())
        df = pd.concat([df, pd.DataFrame([tuple([i, grp[j].key(), grp[j].value()])],
                                         columns=['cluster','seq','id'])], ignore_index=True)

df

Clustering

• Partitioned-based clustering
  • k-means, k-median, fuzzy c-means
• Hierarchical clustering
  • Produces trees of clusters
  • Agglomerative, divisive
  • Advantages
    • It does not require the number of clusters to be specified.
    • Produces a dendrogram which helps with understanding the data.
  • Disadvantages
    • It can never undo any previous steps throughout the algorithm.
    • Sometimes difficult to identify the number of clusters by the dendrogram.
• Density-bassed clustering
  • Produces arbitrary shaped clusters
  • Locates regions of high density, and separates outliers
  • DBSCAN
    
    • Does not require specification of the number of clusters
• Time-series clustering by features 
  • Time-series clustering by features.
    • Raw data.
    • Autocorrelation.
    • Spectral density.
    • Extreme value behavior.
  • Model-based time series clustering.
    • Forecast based clustering.
    • Model with a cluster structure.
  • Time-series clustering by dependence.
• Clustering high dimensional data
  • Many clustering algorithms deal with 1-3 dimensions
  • These methods may not work well when the number of dimensions grows to 20
• Methods for clustering high dimensional data
  • Methods can be grouped into two categories
    • Subspace clustering
      • CLIQUE, ProClus, and bi-clustering approaches
    • Dimensionality reduction approaches
      • Spectral clustering and various dimensionality reduction methods
  • Clustering should not only consider dimensions but also attributes/features
    • Feature selection
    • Feature transformation
      • Principal component analysis, singular value decomposition