7,612
edits
Changes
Jump to navigation
Jump to search
no edit summary
{{Project
|Has project output=Tool
|Has sponsor=Kauffman Incubator Project
|Has title=DSL Encoding
|Has owner=Hiep Nguyen
A sample LSTM cell in tensorflow is as follows:
import tensorflow as tf
def lstm_cell(keep_prob):
'''
'''
if tf.test.is_gpu_available():
lstm = Cudnn_LSTM_with_biastf.contrib.cudnn_rnn.CudnnCompatibleLSTMCell(num_units)#num_units is the number of hidden units in the LSTM cell.
else:
lstm = tf.nn.rnn_cell.LSTMCell(num_units,forget_bias=1.0)
return lstm
Then, we applied a tf.while loop through the cell to build our network. tf.nn.dynamic_rnn will do the work. The sample code is
def lstm_network(x, weightW, biasb,keep_prob):
'''
define stacked cells and prediction
x: data with shape [batch_size,max_len,len_unique_charlen_unique_tokens]
'''
lstm=lstm_cell(keep_prob)
outputs, states = tf.nn.dynamic_rnn(lstm, x, dtype=tf.float32)
prediction = tf.add(tf.matmul(states.h, weightW), biasb,name='prediction')
return prediction
If we want to stack multiple LSTM layers together, we can replace '''lstm=lstm_cell(keep_prob)''' with '''lstm= tf.contrib.rnn.MultiRNNCell([lstm_cell(keep_prob) for _ in range(num_layers)])''' where '''num_layers''' is an integer representing the number of LSTM layers we want
A sample training code lives in
E:\projects\embedding\Web_extractor_model\train_sample.py
In the '''utils.py''' file, there are a few hyperparameters to remember.
max_len: the length of each training point
step: the number of steps we want to move to generate the next training point
num_units: LSTM units, a safe choice is 128
len_unique_chars: total number of unique tokens in all training data
