deephyp.autoencoder.mlp_1D_network

class deephyp.autoencoder.mlp_1D_network(configFile=None, inputSize=None, encoderSize=[50, 30, 10], activationFunc='sigmoid', tiedWeights=None, weightInitOpt='truncated_normal', weightStd=0.1, skipConnect=False, activationFuncFinal='linear')

Class for setting up a 1-D multi-layer perceptron (mlp) autoencoder network. Layers are all fully-connected (i.e. dense).

Parameters:

• configFile (str) – Optional way of setting up the network. All other inputs can be ignored (will be overwritten). Pass the address of the .json config file.
• inputSize (int) – Number of dimensions of input data (i.e. number of spectral bands). Value must be input if not using a config file.
• encoderSize (int list) – Number of nodes at each layer of the encoder. List length is number of encoder layers.
• activationFunc (str) – Activation function for all layers except the last one. Current options: [‘sigmoid’, ‘relu’, ‘linear’].
• tiedWeights (binary list or None) – Specifies whether or not to tie weights at each layer: - 1: tied weights of specific encoder layer to corresponding decoder weights - 0: do not tie weights of specific layer - None: sets all layers to 0
• weightInitOpt (string) – Method of weight initialisation. Current options: [‘gaussian’, ‘truncated_normal’, ‘xavier’, ‘xavier_improved’].
• weightStd (float) – Used by ‘gaussian’ and ‘truncated_normal’ weight initialisation methods.
• skipConnect (boolean) – Whether to use skip connections throughout the network.
• activationFuncFinal (str) – Activation function for final layer. Current options: [‘sigmoid’, ‘relu’, ‘linear’].

inputSize

Number of dimensions of input data (i.e. number of spectral bands).

Type:int

activationFunc

Activation function for all layers except the last one.

Type:str

tiedWeights

Whether (1) or not (0) the weights of an encoder layer are tied to a decoder layer.

Type:binary list

skipConnect

Whether the network uses skip connections between corresponding encoder and decoder layers.

Type:boolean

weightInitOpt

Method of weight initialisation.

Type:string

weightStd

Parameter for ‘gaussian’ and ‘truncated_normal’ weight initialisation methods.

Type:float

activationFuncFinal

Activation function for final layer.

Type:str

encoderSize

Number of inputs and number of nodes at each layer of the encoder.

Type:int list

decoderSize

Number of nodes at each layer of the decoder and number of outputs.

Type:int list

z

Latent representation of data. Accessible through the encoder class function, requiring a trained model.

Type:tensor

y_recon

Reconstructed output of network. Accessible through the decoder and encoder_decoder class functions, requiring a trained model.

Type:tensor

train_ops

Dictionary of names of train and loss ops (suffixed with _train and _loss) added to the network using the add_train_op class function. The name (without suffix) is passed to the train class function to train the network with the referenced train and loss op.

Type:dict

modelsAddrs

Dictionary of model names added to the network using the add_model class function. The names reference models which can be used by the encoder, decoder and encoder_decoder class functions.

Type:dict

add_model(addr, modelName)

Loads a saved set of model parameters for the network.

Parameters:

• addr (str) – Address of the directory containing the checkpoint files.
• modelName (str) – Name of the model (to refer to it later in-case of multiple models for a given network).

add_train_op(name, lossFunc='CSA', learning_rate=0.001, decay_steps=None, decay_rate=None, piecewise_bounds=None, piecewise_values=None, method='Adam', wd_lambda=0.0)

Constructs a loss op and training op from a specific loss function and optimiser. User gives the ops a name, and the train op and loss opp are stored in a dictionary (train_ops) under that name.

Parameters:

• name (str) – Name of the training op (to refer to it later in-case of multiple training ops).
• lossFunc (str) – Reconstruction loss function.
• learning_rate (float) – Controls the degree to which the weights are updated during training.
• decay_steps (int) – Epoch frequency at which to decay the learning rate.
• decay_rate (float) – Fraction at which to decay the learning rate.
• piecewise_bounds (int list) – Epoch step intervals for decaying the learning rate. Alternative to decay steps.
• piecewise_values (float list) – Rate at which to decay the learning rate at the piecewise_bounds.
• method (str) – Optimisation method.
• wd_lambda (float) – Scalar to control weighting of weight decay in loss.

decoder(modelName, dataZ)

Extract the reconstruction of some dataSamples from their latent representation encoding using a trained model.

Parameters:

• modelName (str) – Name of the model to use (previously added with add_model() ).
• dataZ (np.array) – Latent representation of data samples to reconstruct using the network. Shape [numSamples x arbitrary].

Returns:

Reconstructed data (y_recon attribute). Shape [numSamples x arbitrary].

Return type:

(np.array)

encoder(modelName, dataSamples)

Extract the latent variable of some dataSamples using a trained model.

Parameters:

• modelName (str) – Name of the model to use (previously added with add_model() ).
• dataSample (np.array) – Shape [numSamples x inputSize].

Returns:

Latent representation z of dataSamples. Shape [numSamples x arbitrary].

Return type:

(np.array)

encoder_decoder(modelName, dataSamples)

Extract the reconstruction of some dataSamples using a trained model.

Parameters:

• modelName (str) – Name of the model to use (previously added with add_model() ).
• dataSample (np.array) – Data samples to reconstruct using the network. Shape [numSamples x inputSize].

Returns:

Reconstructed data (y_recon attribute). Shape [numSamples x arbitrary].

Return type:

(np.array)

train(dataTrain, dataVal, train_op_name, n_epochs, save_addr, visualiseRateTrain=0, visualiseRateVal=0, save_epochs=[1000])

Calls network_ops function to train a network.

Parameters:

• dataTrain (obj) – Iterator object for training data.
• dataVal (obj) – Iterator object for validation data.
• train_op_name (str) – Name of training op created.
• n_epochs (int) – Number of loops through dataset to train for.
• save_addr (str) – Address of a directory to save checkpoints for desired epochs, or address of saved checkpoint. If address is for an epoch and contains a previously saved checkpoint, then the network will start training from there. Otherwise it will be trained from scratch.
• visualiseRateTrain (int) – Epoch rate at which to print training loss in console.
• visualiseRateVal (int) – Epoch rate at which to print validation loss in console.
• save_epochs (int list) – Epochs to save checkpoints at.