deephyp package

Submodules

deephyp.autoencoder module

Description: high-level deep learning classes for building, training and using unsupervised autoencoders. Uses functions from the low-level network_ops module.

  • File name: autoencoder.py
  • Author: Lloyd Windrim
  • Date created: June 2019
  • Python package: deephyp
class deephyp.autoencoder.cnn_1D_network(configFile=None, inputSize=None, zDim=5, encoderNumFilters=[10, 10, 10], encoderFilterSize=[20, 10, 10], activationFunc='sigmoid', tiedWeights=None, weightInitOpt='truncated_normal', weightStd=0.1, skipConnect=False, padding='VALID', encoderStride=[1, 1, 1], activationFuncFinal='linear')[source]

Bases: object

Class for setting up a 1-D convolutional autoencoder network. Builds a network with an encoder containing convolutional layers followed by a single fully-connected layer to map from the final convolutional layer in the encoder to the latent layer. The decoder contains a single fully-connected layer and then several deconvolutional layers which reconstruct the spectra in the output.

Parameters:
  • configFile (str) – Optional way of setting up the network. All other inputs can be ignored (will be overwritten). Pass the address of the .json config file.
  • inputSize (int) – Number of dimensions of input data (i.e. number of spectral bands). Value must be input if not using a config file.
  • zDim (int) – Dimensionality of latent vector.
  • encoderNumFilters (int list) – Number of filters at each layer of the encoder. List length is number of convolutional encoder layers. Note that there is a single mlp layer after the last convolutional layer.
  • encoderFilterSize (int list) – Size of filter at each layer of the encoder. List length is number of encoder layers.
  • activationFunc (str) – Activation function for all layers except the last one. Current options: [‘sigmoid’, ‘relu’, ‘linear’].
  • tiedWeights (binary list or None) – Specifies whether or not to tie weights at each layer: - 1: tied weights of specific encoder layer to corresponding decoder weights - 0: do not tie weights of specific layer - None: sets all layers to 0
  • weightInitOpt (string) – Method of weight initialisation. Current options: [‘gaussian’, ‘truncated_normal’, ‘xavier’, ‘xavier_improved’].
  • weightStd (float) – Used by ‘gaussian’ and ‘truncated_normal’ weight initialisation methods.
  • skipConnect (boolean) – Whether to use skip connections throughout the network.
  • padding (str) – Type of padding used. Current options: [‘VALID’, ‘SAME’].
  • encoderStride (int list) – Stride at each convolutional encoder layer.
  • activationFuncFinal (str) – Activation function for final layer. Current options: [‘sigmoid’, ‘relu’, ‘linear’].
inputSize

Number of dimensions of input data (i.e. number of spectral bands).

Type:int
activationFunc

Activation function for all layers except the last one.

Type:str
tiedWeights

Whether (1) or not (0) the weights of an encoder layer are tied to a decoder layer.

Type:binary list
skipConnect

Whether the network uses skip connections between corresponding encoder and decoder layers.

Type:boolean
weightInitOpt

Method of weight initialisation.

Type:string
weightStd

Parameter for ‘gaussian’ and ‘truncated_normal’ weight initialisation methods.

Type:float
activationFuncFinal

Activation function for final layer.

Type:str
encoderNumFilters

Number of filters at each layer of the encoder. List length is number of convolutional encoder layers. Note that there is a single mlp layer after the last convolutional layer.

Type:int list
encoderFilterSize

Size of filter at each layer of the encoder. List length is number of encoder layers.

Type:int list
encoderStride

Stride at each convolutional encoder layer.

Type:int list
decoderNumFilters
Type:int list
decoderFilterSize
Type:int list
decoderStride
Type:int list
zDim

Dimensionality of latent vector.

Type:int
padding

Type of padding used. Current options: [‘VALID’, ‘SAME’].

Type:str
z

Latent representation of data. Accessible through the encoder class function, requiring a trained model.

Type:tensor
y_recon

Reconstructed output of network. Accessible through the decoder and encoder_decoder class functions, requiring a trained model.

Type:tensor
train_ops

Dictionary of names of train and loss ops (suffixed with _train and _loss) added to the network using the add_train_op class function. The name (without suffix) is passed to the train class function to train the network with the referenced train and loss op.

Type:dict
modelsAddrs

Dictionary of model names added to the network using the add_model class function. The names reference models which can be used by the encoder, decoder and encoder_decoder class functions.

Type:dict
add_model(addr, modelName)[source]

Loads a saved set of model parameters for the network.

Parameters:
  • addr (str) – Address of the directory containing the checkpoint files.
  • modelName (str) – Name of the model (to refer to it later in-case of multiple models for a given network).
add_train_op(name, lossFunc='SSE', learning_rate=0.001, decay_steps=None, decay_rate=None, piecewise_bounds=None, piecewise_values=None, method='Adam', wd_lambda=0.0)[source]

Constructs a loss op and training op from a specific loss function and optimiser. User gives the ops a name, and the train op and loss opp are stored in a dictionary (train_ops) under that name.

Parameters:
  • name (str) – Name of the training op (to refer to it later in-case of multiple training ops).
  • lossFunc (str) – Reconstruction loss function.
  • learning_rate (float) – Controls the degree to which the weights are updated during training.
  • decay_steps (int) – Epoch frequency at which to decay the learning rate.
  • decay_rate (float) – Fraction at which to decay the learning rate.
  • piecewise_bounds (int list) – Epoch step intervals for decaying the learning rate. Alternative to decay steps.
  • piecewise_values (float list) – Rate at which to decay the learning rate at the piecewise_bounds.
  • method (str) – Optimisation method.
  • wd_lambda (float) – Scalar to control weighting of weight decay in loss.
decoder(modelName, dataZ)[source]

Extract the reconstruction of some dataSamples from their latent representation encoding using a trained model.

Parameters:
  • modelName (str) – Name of the model to use (previously added with add_model() ).
  • dataZ (np.array) – Latent representation of data samples to reconstruct using the network. Shape [numSamples x arbitrary].
Returns:

Reconstructed data (y_recon attribute). Shape [numSamples x arbitrary].
Return type:

(np.array)
encoder(modelName, dataSamples)[source]

Extract the latent variable of some dataSamples using a trained model.

Parameters:
  • modelName (str) – Name of the model to use (previously added with add_model() ).
  • dataSample (np.array) – Shape [numSamples x inputSize].
Returns:

Latent representation z of dataSamples. Shape [numSamples x arbitrary].
Return type:

(np.array)
encoder_decoder(modelName, dataSamples)[source]

Extract the reconstruction of some dataSamples using a trained model.

Parameters:
  • modelName (str) – Name of the model to use (previously added with add_model() ).
  • dataSample (np.array) – Data samples to reconstruct using the network. Shape [numSamples x inputSize].
Returns:

Reconstructed data (y_recon attribute). Shape [numSamples x arbitrary].
Return type:

(np.array)
train(dataTrain, dataVal, train_op_name, n_epochs, save_addr, visualiseRateTrain=0, visualiseRateVal=0, save_epochs=[1000])[source]

Calls network_ops function to train a network.

Parameters:
  • dataTrain (obj) – Iterator object for training data.
  • dataVal (obj) – Iterator object for validation data.
  • train_op_name (str) – Name of training op created.
  • n_epochs (int) – Number of loops through dataset to train for.
  • save_addr (str) – Address of a directory to save checkpoints for desired epochs, or address of saved checkpoint. If address is for an epoch and contains a previously saved checkpoint, then the network will start training from there. Otherwise it will be trained from scratch.
  • visualiseRateTrain (int) – Epoch rate at which to print training loss in console.
  • visualiseRateVal (int) – Epoch rate at which to print validation loss in console.
  • save_epochs (int list) – Epochs to save checkpoints at.
class deephyp.autoencoder.mlp_1D_network(configFile=None, inputSize=None, encoderSize=[50, 30, 10], activationFunc='sigmoid', tiedWeights=None, weightInitOpt='truncated_normal', weightStd=0.1, skipConnect=False, activationFuncFinal='linear')[source]

Bases: object

Class for setting up a 1-D multi-layer perceptron (mlp) autoencoder network. Layers are all fully-connected (i.e. dense).

Parameters:
  • configFile (str) – Optional way of setting up the network. All other inputs can be ignored (will be overwritten). Pass the address of the .json config file.
  • inputSize (int) – Number of dimensions of input data (i.e. number of spectral bands). Value must be input if not using a config file.
  • encoderSize (int list) – Number of nodes at each layer of the encoder. List length is number of encoder layers.
  • activationFunc (str) – Activation function for all layers except the last one. Current options: [‘sigmoid’, ‘relu’, ‘linear’].
  • tiedWeights (binary list or None) – Specifies whether or not to tie weights at each layer: - 1: tied weights of specific encoder layer to corresponding decoder weights - 0: do not tie weights of specific layer - None: sets all layers to 0
  • weightInitOpt (string) – Method of weight initialisation. Current options: [‘gaussian’, ‘truncated_normal’, ‘xavier’, ‘xavier_improved’].
  • weightStd (float) – Used by ‘gaussian’ and ‘truncated_normal’ weight initialisation methods.
  • skipConnect (boolean) – Whether to use skip connections throughout the network.
  • activationFuncFinal (str) – Activation function for final layer. Current options: [‘sigmoid’, ‘relu’, ‘linear’].
inputSize

Number of dimensions of input data (i.e. number of spectral bands).

Type:int
activationFunc

Activation function for all layers except the last one.

Type:str
tiedWeights

Whether (1) or not (0) the weights of an encoder layer are tied to a decoder layer.

Type:binary list
skipConnect

Whether the network uses skip connections between corresponding encoder and decoder layers.

Type:boolean
weightInitOpt

Method of weight initialisation.

Type:string
weightStd

Parameter for ‘gaussian’ and ‘truncated_normal’ weight initialisation methods.

Type:float
activationFuncFinal

Activation function for final layer.

Type:str
encoderSize

Number of inputs and number of nodes at each layer of the encoder.

Type:int list
decoderSize

Number of nodes at each layer of the decoder and number of outputs.

Type:int list
z

Latent representation of data. Accessible through the encoder class function, requiring a trained model.

Type:tensor
y_recon

Reconstructed output of network. Accessible through the decoder and encoder_decoder class functions, requiring a trained model.

Type:tensor
train_ops

Dictionary of names of train and loss ops (suffixed with _train and _loss) added to the network using the add_train_op class function. The name (without suffix) is passed to the train class function to train the network with the referenced train and loss op.

Type:dict
modelsAddrs

Dictionary of model names added to the network using the add_model class function. The names reference models which can be used by the encoder, decoder and encoder_decoder class functions.

Type:dict
add_model(addr, modelName)[source]

Loads a saved set of model parameters for the network.

Parameters:
  • addr (str) – Address of the directory containing the checkpoint files.
  • modelName (str) – Name of the model (to refer to it later in-case of multiple models for a given network).
add_train_op(name, lossFunc='CSA', learning_rate=0.001, decay_steps=None, decay_rate=None, piecewise_bounds=None, piecewise_values=None, method='Adam', wd_lambda=0.0)[source]

Constructs a loss op and training op from a specific loss function and optimiser. User gives the ops a name, and the train op and loss opp are stored in a dictionary (train_ops) under that name.

Parameters:
  • name (str) – Name of the training op (to refer to it later in-case of multiple training ops).
  • lossFunc (str) – Reconstruction loss function.
  • learning_rate (float) – Controls the degree to which the weights are updated during training.
  • decay_steps (int) – Epoch frequency at which to decay the learning rate.
  • decay_rate (float) – Fraction at which to decay the learning rate.
  • piecewise_bounds (int list) – Epoch step intervals for decaying the learning rate. Alternative to decay steps.
  • piecewise_values (float list) – Rate at which to decay the learning rate at the piecewise_bounds.
  • method (str) – Optimisation method.
  • wd_lambda (float) – Scalar to control weighting of weight decay in loss.
decoder(modelName, dataZ)[source]

Extract the reconstruction of some dataSamples from their latent representation encoding using a trained model.

Parameters:
  • modelName (str) – Name of the model to use (previously added with add_model() ).
  • dataZ (np.array) – Latent representation of data samples to reconstruct using the network. Shape [numSamples x arbitrary].
Returns:

Reconstructed data (y_recon attribute). Shape [numSamples x arbitrary].
Return type:

(np.array)
encoder(modelName, dataSamples)[source]

Extract the latent variable of some dataSamples using a trained model.

Parameters:
  • modelName (str) – Name of the model to use (previously added with add_model() ).
  • dataSample (np.array) – Shape [numSamples x inputSize].
Returns:

Latent representation z of dataSamples. Shape [numSamples x arbitrary].
Return type:

(np.array)
encoder_decoder(modelName, dataSamples)[source]

Extract the reconstruction of some dataSamples using a trained model.

Parameters:
  • modelName (str) – Name of the model to use (previously added with add_model() ).
  • dataSample (np.array) – Data samples to reconstruct using the network. Shape [numSamples x inputSize].
Returns:

Reconstructed data (y_recon attribute). Shape [numSamples x arbitrary].
Return type:

(np.array)
train(dataTrain, dataVal, train_op_name, n_epochs, save_addr, visualiseRateTrain=0, visualiseRateVal=0, save_epochs=[1000])[source]

Calls network_ops function to train a network.

Parameters:
  • dataTrain (obj) – Iterator object for training data.
  • dataVal (obj) – Iterator object for validation data.
  • train_op_name (str) – Name of training op created.
  • n_epochs (int) – Number of loops through dataset to train for.
  • save_addr (str) – Address of a directory to save checkpoints for desired epochs, or address of saved checkpoint. If address is for an epoch and contains a previously saved checkpoint, then the network will start training from there. Otherwise it will be trained from scratch.
  • visualiseRateTrain (int) – Epoch rate at which to print training loss in console.
  • visualiseRateVal (int) – Epoch rate at which to print validation loss in console.
  • save_epochs (int list) – Epochs to save checkpoints at.

deephyp.classifier module

Description: high-level deep learning classes for building, training and using supervised neural network classifiers. Uses functions from the low-level network_ops module.

  • File name: classifier.py
  • Author: Lloyd Windrim
  • Date created: August 2019
  • Python package: deephyp
class deephyp.classifier.cnn_1D_network(configFile=None, inputSize=None, numClasses=None, convFilterSize=[20, 10, 10], convNumFilters=[10, 10, 10], convStride=[1, 1, 1], fcSize=[20, 20], activationFunc='relu', weightInitOpt='truncated_normal', weightStd=0.1, padding='VALID')[source]

Bases: object

Class for setting up a 1-D convolutional neural network (cnn) for classification. Contains several convolutional layers followed by several fully-connected layers. The network outputs scores for each class, for a given set of input data samples.

Parameters:
  • configFile (str) – Optional way of setting up the network. All other inputs can be ignored (will be overwritten). Pass the address of the .json config file.
  • inputSize (int) – Number of dimensions of input data (i.e. number of spectral bands). Value must be input if not using a config file.
  • numClasses (int) – Number of labelled classes in the dataset (not including the zero class).
  • convFilterSize (int list) – Size of filter at each convolutional layer. List length is number of convolutional layers.
  • convNumFilters (int list) – Number of filters at each convolutional layer of the network. List length is number of convolutional layers.
  • convStride (int list) – Stride at each convolutional layer. List length is number of convolutional layers. fcSize (int list): Number of nodes at each fully-connected (i.e. dense) layer of the encoder. List length is number of fully-connected layers.
  • activationFunc (str) – Activation function for all layers except the last one. Current options: [‘sigmoid’, ‘relu’, ‘linear’].
  • weightInitOpt (string) – Method of weight initialisation. Current options: [‘gaussian’, ‘truncated_normal’, ‘xavier’, ‘xavier_improved’].
  • weightStd (float) – Used by ‘gaussian’ and ‘truncated_normal’ weight initialisation methods.
  • padding (str) – Type of padding used. Current options: [‘VALID’, ‘SAME’].
inputSize

Number of dimensions of input data (i.e. number of spectral bands).

Type:int
activationFunc

Activation function for all layers except the last one.

Type:str
weightInitOpt

Method of weight initialisation.

Type:string
weightStd

Parameter for ‘gaussian’ and ‘truncated_normal’ weight initialisation methods.

Type:float
convFilterSize

Size of filter at each convolutional layer. List length is number of convolutional layers.

Type:int list
convNumFilters

Number of filters at each convolutional layer of the network. List length is number of convolutional layers.

Type:int list
convStride

Stride at each convolutional layer. List length is number of convolutional layers. padding (str): Type of padding used. Current options: [‘VALID’, ‘SAME’].

Type:int list
fcSize

Number of nodes at each fully-connected (i.e. dense) layer of the encoder. List length is number of fully-connected layers.

Type:int list
numLayers

Total number of layers (convolutional and fully-connected).

Type:int
y_pred

Output of network - class scores with shape [numSamples x numClasses]. Accessible through the predict_scores class functions, requiring a trained model.

Type:tensor
train_ops

Dictionary of names of train and loss ops (suffixed with _train and _loss) added to the network using the add_train_op class function. The name (without suffix) is passed to the train class function to train the network with the referenced train and loss op.

Type:dict
modelsAddrs

Dictionary of model names added to the network using the add_model class function. The names reference models which can be used by the predict_scores, predict_labels and predict_features class functions.

Type:dict
add_model(addr, modelName)[source]

Loads a saved set of model parameters for the network.

Parameters:
  • addr (str) – Address of the directory containing the checkpoint files.
  • modelName (str) – Name of the model (to refer to it later in-case of multiple models for a given network).
add_train_op(name, balance_classes=True, learning_rate=0.001, decay_steps=None, decay_rate=None, piecewise_bounds=None, piecewise_values=None, method='Adam', wd_lambda=0.0)[source]

Constructs a loss op and training op from a specific loss function and optimiser. User gives the ops a name, and the train op and loss opp are stored in a dictionary (train_ops) under that name.

Parameters:
  • name (str) – Name of the training op (to refer to it later in-case of multiple training ops).
  • balance_classes (boolean) – Weight the samples during training so that the contribtion to the loss of each class is balanced by the number of samples the class has (in a given batch).
  • learning_rate (float) – Controls the degree to which the weights are updated during training.
  • decay_steps (int) – Epoch frequency at which to decay the learning rate.
  • decay_rate (float) – Fraction at which to decay the learning rate.
  • piecewise_bounds (int list) – Epoch step intervals for decaying the learning rate. Alternative to decay steps.
  • piecewise_values (float list) – Rate at which to decay the learning rate at the piecewise_bounds.
  • method (str) – Optimisation method.
  • wd_lambda (float) – Scalar to control weighting of weight decay in loss.
predict_features(modelName, dataSamples, layer)[source]

Extract the predicted feature values at a particular layer of the network.

Parameters:
  • modelName (str) – Name of the model to use (previously added with add_model() )
  • dataSamples (np.array) – Shape [numSamples x inputSize]
  • layer (int) – Layer at which to extract features. Must be between 1 and numLayers inclusive.
Returns:

Values of neurons at layer. Shape [numSamples x numNeurons] if fully-connected layer and [numSamples x convDim1 x convDim2] if convolutional layer.
Return type:

(np.array)
predict_labels(modelName, dataSamples)[source]

Extract the predicted classification labels of some dataSamples using a trained model.

Parameters:
  • modelName (str) – Name of the model to use (previously added with add_model() )
  • dataSamples (array) – Shape [numSamples x inputSize]
Returns:

Predicted classification labels of dataSamples. Shape [numSamples].
Return type:

(np.array)
predict_scores(modelName, dataSamples, useSoftmax=True)[source]

Extract the predicted classification scores of some dataSamples using a trained model.

Parameters:
  • modelName (str) – Name of the model to use (previously added with add_model() ).
  • dataSamples (np.array) – Shape [numSamples x inputSize].
  • useSoftmax (boolean) – Pass predicted scores output by network through a softmax function.
Returns:

Predicted classification scores of dataSamples. Shape [numSamples x numClasses].
Return type:

(np.array)
train(dataTrain, dataVal, train_op_name, n_epochs, save_addr, visualiseRateTrain=0, visualiseRateVal=0, save_epochs=[1000])[source]

Calls network_ops function to train a network.

Parameters:
  • dataTrain (obj) – Iterator object for training data.
  • dataVal (obj) – Iterator object for validation data.
  • train_op_name (str) – Name of training op created.
  • n_epochs (int) – Number of loops through dataset to train for.
  • save_addr (str) – Address of a directory to save checkpoints for desired epochs, or address of saved checkpoint. If address is for an epoch and contains a previously saved checkpoint, then the network will start training from there. Otherwise it will be trained from scratch.
  • visualiseRateTrain (int) – Epoch rate at which to print training loss in console.
  • visualiseRateVal (int) – Epoch rate at which to print validation loss in console.
  • save_epochs (int list) – Epochs to save checkpoints at.

deephyp.data module

Description: high-level classes for using hyperspectral data with the deep learning modules.

  • File name: data.py
  • Author: Lloyd Windrim
  • Date created: June 2019
  • Python package: deephyp
class deephyp.data.HypImg(spectralInput, labels=None, wavelengths=None, bands=None)[source]

Bases: object

Class for handling data. Stores meta-data and contains attributes for pre-processing the data. If passed labels, samples with label zero are considered as a background class. This class is not included in numClasses and data samples with this label have a one-hot vector label of all zeros.

Parameters:
  • spectralInput (np.array float) – Spectral dataset. Shape can be [numRows x numCols x numBands] or [numSamples x numBands].
  • wavelengths (np.array float) – Vector of wavelengths that spectralInput wavelengths lie within.
  • bands (np.array int) – Wavelength indexes for each band of spectralInput. Shape [numBands].
  • labels (np.array int) – Class labels for each spectral sample in spectralInput. Shape can be [numRows x numCols] or [numSamples].
spectra

Un-pre-processed spectral data with shape [numSamples x numBands].

Type:np.array float
spectraCube

If data passed as image - un-pre-processed spectral datacube with shape [numSamples x numBands]. Else None.

Type:np.array float
spectraPrep

Pre-processed spectral data with shape [numSamples x numBands].

Type:np.array float
numSamples

The number of spectra.

Type:int
numRows

If data passed as image - the number of image rows. Else None.

Type:int
numCols

If data passed as image - the number of image columns. Else None.

Type:int
wavelengths

If provided - vector of wavelengths that spectra wavelengths lie within. Else None.

Type:np.array float
bands

If provided - wavelength indexes for each band of spectra with shape [numBands]. Else None.

Type:np.array int
labels

If provided - class labels for each spectral sample with shape [numSamples]. Else None.

Type:np.array int
labelsOnehot

If labels provided - the one-hot label vector for each sample. Samples with label zero (background class) have a one-hot vector of all zeros. Else None.

Type:np.array int
pre_process(method='minmax')[source]

Pre-process data for input into the network. Stores in the spectraPrep attribute.

Parameters:method (str) – Method of pre-processing. Current options: ‘minmax’
class deephyp.data.Iterator(dataSamples, targets, batchSize=None)[source]

Bases: object

Class for iterating through data, to train the network.

Parameters:
  • dataSamples (np.array float) – Data to be input into the network. Shape [numSamples x numBands].
  • targets (np.array int) – Network output target of each dataSample. For classification, these are the class labels, and it could be the dataSamples for autoencoders. Shape [numSamples x arbitrary]
  • batchSize (int) – Number of dataSamples per batch
dataSamples

Data to be input into the network. Shape [numSamples x numBands].

Type:np.array float
targets

Network output target of each dataSample. For classification, these are the class labels, and it could be the dataSamples for autoencoders. Shape [numSamples x arbitrary]

Type:np.array int
batchSize

Number of dataSamples per batch. If None - set to numSamples (i.e. whole dataset).

Type:int
numSamples

The number of data samples.

Type:int
currentBatch

A list of indexes specifying the data samples in the current batch. Shape [batchSize]

Type:int list
get_batch(idx)[source]

Returns a specified set of samples and targets.

Parameters:idx (int list) – Indexes of samples (and targets) to return.
Returns:2-element tuple containing:
  • (np.array float) - Batch of data samples at [idx] indexes. Shape [length(idx) x numBands].
  • (np.array int) - Batch of targets at [idx] indexes. Shape [length(idx) x arbitrary].
Return type:(tuple)
next_batch()[source]

Return next batch of samples and targets (with batchSize number of samples). The currentBatch indexes are incremented. If end of dataset reached, the indexes wraps around to the beginning.

Returns:2-element tuple containing:
  • (np.array float) - Batch of data samples at currentBatch indexes. Shape [batchSize x numBands].
  • (np.array int) - Batch of targets at currentBatch indexes. Shape [batchSize x arbitrary].
Return type:(tuple)
reset_batch()[source]

Resets the current batch to the beginning.

shuffle()[source]

Randomly permutes all dataSamples (and corresponding targets).

deephyp.network_ops module

Description: various functions for deep learning built on-top of tensorflow. The high-level modules in the package call these functions.

  • File name: network_ops.py
  • Author: Lloyd Windrim
  • Date created: June 2019
  • Python package: deephyp
deephyp.network_ops.balance_classes(y_target, num_classes)[source]

Calculates the class weights needed to balance the classes, based on the number of samples of each class in the batch of data.

Parameters:
  • y_target (tensor) – One-hot classification labels (1D vector). Shape [numSamples x numClasses]
  • num_classes (int) –
Returns:

A weighting for each class that balances their contribution to the loss. Shape [numClasses].
Return type:

(tensor)
deephyp.network_ops.conv_output_shape(inputShape, filterSize, padding, stride)[source]

Computes the expected output shape (for the convolving axis only) of a convolution layer given an input shape.

Parameters:
  • inputShape (int) – Shape of convolving axis of input data.
  • filterSize (int) – Size of filter/kernel of convolution layer.
  • stride (int) – Stride at which to convolve (must be >= 1).
  • padding (str) – Type of padding to use (‘SAME’ or ‘VALID’).
Returns:

Output shape of convolving axis for given layer and input shape.
Return type:

(int)
deephyp.network_ops.create_variable(shape, method='gaussian', wd=False)[source]

Setup a trainable variable (collection of parameters) of a particular shape.

Parameters:
  • shape (list) – Data shape.
  • method (str) – How to initialise parameter values.
  • wd (boolean) – Setup weight decay for this variable.
Returns:

Set of parameters for the given variable.
Return type:

(tensor)
deephyp.network_ops.init_weight(opts, shape, stddev=0.1, const=0.1, wd=False, dtype=tf.float32)[source]

Weight initialisation function.

Parameters:
  • opts (str) – Method for initialising variable. (‘gaussian’,’truncated_normal’,’xavier’,’xavier_improved’, ‘constant’).
  • shape (list) – Data shape.
  • stddev (int) – Standard deviation used by ‘gaussian’ and ‘truncated_normal’ variable initialisation methods.
  • const (int) – Constant value to initialise variable to if using ‘constant’ method.
  • wd (boolean) – Whether this variable contributes to weight decay or not.
  • dtype (tf.dtype) – Data type for variable.
Returns:
Return type:

weights
deephyp.network_ops.layer_activation(input, func='sigmoid')[source]

Define an activation function operation.

Parameters:
  • input (tensor) – Data input into the function.
  • func (str) – Type of activation function. (relu, sigmoid, linear).
Returns:

Computes activation. Shape is same as input.
Return type:

(tensor)
deephyp.network_ops.layer_conv1d(input, W, b, stride=1, padding='SAME')[source]

Define a 1 dimensional convolution layer operation.

Parameters:
  • input (tensor) – Data input into the layer. Shape [numSamples x numInputNeurons x numFiltersIn].
  • W (tensor) – Weight parameters of the filters/kernels. Shape [filterSize x numFiltersIn x numFiltersOut].
  • b (tensor) – Bias parameters for the layer. Shape [numFiltersOut].
  • stride (int) – Stride at which to convolve (must be >= 1).
  • padding (str) – Type of padding to use (‘SAME’ or ‘VALID’).
Returns:

Computes layer output. Shape [numSamples x numOutputNeurons x numFiltersOut].
Return type:

(tensor)
deephyp.network_ops.layer_deconv1d(input, W, b, outputShape, stride=1, padding='SAME')[source]

Define a 1 dimensional deconvolution layer operation. Also called convolutional transpose or upsampling layer.

Parameters:
  • input (tensor) – Data input into the layer. Shape [numSamples x numInputNeurons x numFiltersIn].
  • W (tensor) – Weight parameters of the filters/kernels. Shape [filterSize x numFiltersOut x numFiltersIn].
  • b (tensor) – Bias parameters for the layer. Shape [numFiltersOut].
  • outputShape (list) – Expected shape of the layer output. Shape [numSamples x numOutputNeurons x numFiltersOut].
  • stride (int) – Stride at which to convolve (must be >= 1).
  • padding (str) – Type of padding to use (‘SAME’ or ‘VALID’).
Returns:

Computes layer output. Shape [numSamples x numOutputNeurons x numFiltersOut].
Return type:

(tensor)
deephyp.network_ops.layer_fullyConn(input, W, b)[source]

Define a fully connected layer operation. Also called a ‘dense’ layer.

Parameters:
  • input (tensor) – Data input into the layer. Shape [numSamples x numInputNeurons].
  • W (tensor) – Weight parameters for the layer. Shape [numInputNeurons x numOutputNeurons].
  • b (tensor) – Bias parameters for the layer. Shape [numOutputNeurons].
Returns:

Computes layer output. Shape [numSamples x numOutputNeurons].
Return type:

(tensor)
deephyp.network_ops.load_config(net_obj, addr)[source]

Loads a network config file. Loads from variables in the config.json file and overwrites variables in network object. Applies to variables in the net_config list in the network object.

Parameters:
  • net_obj (obj) – Network object.
  • addr (obj) – Directory location of config.json file.
deephyp.network_ops.load_model(addr, sess)[source]

Loads a model from the address of a checkpoint.

Parameters:
  • addr (str) – Address of a directory to save checkpoint for current epoch.
  • sess (obj) – Tensor flow session object.
deephyp.network_ops.loss_function_crossentropy_1D(y_pred, y_target, class_weights=None, num_classes=None)[source]

Cross entropy loss function op, comparing 1D tensors for network prediction and target. Weights the classes when calculating the loss to balance un-even training batches. If class weights are not provided, then no weighting is done (weight of 1 assigned to each class).

Parameters:
  • y_pred (tensor) – Output of network (1D vector of class scores). Shape [numSamples x numClasses].
  • y_target (tensor) – One-hot classification labels (1D vector). Shape [numSamples x numClasses].
  • class_weights (tensor) – Weight for each class. Shape [numClasses].
  • num_classes (int) –
Returns:

Cross-entropy loss.
Return type:

(tensor)
deephyp.network_ops.loss_function_reconstruction_1D(y_reconstructed, y_target, func='SSE')[source]

Reconstruction loss function op, comparing 1D tensors for network reconstruction and target.

Parameters:
  • y_reconstructed (tensor) – Output of network (reconstructed 1D vector). Shape [numSamples x inputSize].
  • y_target (tensor) – What the network is trying to reconstruct (1D vector). Shape [numSamples x inputSize].
  • func (string) – The name of the loss function to be used. ‘SSE’-sum of square errors,’CSA’-cosine spectral angle, ‘SA’-spectral angle, ‘SID’-spectral information divergence.
Returns:

Reconstruction loss.
Return type:

(tensor)
deephyp.network_ops.loss_weight_decay(wdLambda)[source]

Weight decay loss op, regularises network by penalising parameters for being too large.

Parameters:wdLambda (float) – Scalar to control weighting of weight decay in loss.
Returns:Weight-decay loss.
Return type:(tensor)
deephyp.network_ops.save_config(net_obj, addr)[source]

Saves a network config file. Saves the variables listed in net_config within the network object.

Parameters:
  • net_obj (obj) – Network object.
  • addr (obj) – Directory of where to store the config.json file.
deephyp.network_ops.save_model(addr, sess, saver, current_epoch, epochs_to_save)[source]

Saves a checkpoint at a list of epochs.

Parameters:
  • addr (str) – Address of a directory to save checkpoint for current epoch.
  • sess (obj) – Tensor flow session object.
  • saver (obj) – Tensor flow save object.
  • current_epoch (int) – The current epoch.
  • epochs_to_save (int list) – Epochs to save checkpoints at.
deephyp.network_ops.train(net_obj, dataTrain, dataVal, train_op_name, n_epochs, save_addr, visualiseRateTrain=0, visualiseRateVal=0, save_epochs=[1000])[source]

Function for training a network. Updates the network weights through the training op. The function will check the save address for a model checkpoint to load, otherwise it will begin training from scratch.

Parameters:
  • net_obj (obj) – Network object.
  • dataTrain (obj) – Iterator object for training data.
  • dataVal (obj) – Iterator object for validation data.
  • train_op_name (string) – Name of training op created.
  • n_epochs (int) – Number of loops through dataset to train for.
  • save_addr (str) – Address of a directory to save checkpoints for desired epochs, or address of saved checkpoint. If address is for an epoch and contains a previously saved checkpoint, then the network will start training from there. Otherwise it will be trained from scratch.
  • visualiseRateTrain (int) – Epoch rate at which to print training loss in console.
  • visualiseRateVal (int) – Epoch rate at which to print validation loss in console.
  • save_epochs (int list) – Epochs to save checkpoints at.
deephyp.network_ops.train_step(loss, learning_rate=0.001, decay_steps=None, decay_rate=None, piecewise_bounds=None, piecewise_values=None, method='Adam')[source]

Operation for training the weights of the network by optimising them to minimise the loss function. Note that the default is a constant learning rate (no decay).

Parameters:
  • loss (tensor) – Output of network loss function.
  • learning_rate – (float) Controls the degree to which the weights are updated during training.
  • decay_steps (int) – Epoch frequency at which to decay the learning rate.
  • decay_rate (float) – Fraction at which to decay the learning rate.
  • piecewise_bounds (int list) – Epoch step intervals for decaying the learning rate. Alternative to decay steps.
  • piecewise_values (float list) – Rate at which to decay the learning rate at the piecewise_bounds.
  • method (str) – Optimisation method. (Adam, SGD).
Returns:

(op) A train op.

Module contents