Coding the forward propagation algorithm part 1

APUNTES

COMPLETE CODES DEEP LEARNING BASIS HERE:

https://physicsphd.blogspot.com/2019/01/introduction-to-deep-learning.html

Coding the forward propagation algorithm

In this exercise, you'll write code to do forward propagation (prediction) for your first neural network:

Each data point is a customer. The first input is how many accounts they have, and the second input is how many children they have. The model will predict how many transactions the user makes in the next year. You will use this data throughout the first 2 chapters of this course.

The input data has been pre-loaded as input_data, and the weights are available in a dictionary called weights. The array of weights for the first node in the hidden layer are in weights['node_0'], and the array of weights for the second node in the hidden layer are in weights['node_1'].

The weights feeding into the output node are available in weights['output'].

NumPy will be pre-imported for you as np in all exercises.

Instructions

100 XP

Instructions

100 XP

• Calculate the value in node 0 by multiplying input_data by its weights weights['node_0'] and computing their sum. This is the 1st node in the hidden layer.
• Calculate the value in node 1 using input_data and weights['node_1']. This is the 2nd node in the hidden layer.
• Put the hidden layer values into an array. This has been done for you.
• Generate the prediction by multiplying hidden_layer_outputsby weights['output'] and computing their sum.
• Hit 'Submit Answer' to print the output! 

Take Hint (-30 XP)

            

                

# Calculate node 0 value: node_0_value

node_0_value = (input_data*weights['node_0']).sum()

# Calculate node 1 value: node_1_value

node_1_value = (input_data*weights['node_1']).sum()

# Put node values into array: hidden_layer_outputs

hidden_layer_outputs = np.array([node_0_value, node_1_value])

# Calculate output: output

output = (hidden_layer_outputs*weights['output']).sum()

# Print output

print(output)

# Calculate node 0 value: node_0_value

node_0_value = (input_data*weights['node_0']).sum()

# Calculate node 1 value: node_1_value

node_1_value = (input_data*weights['node_1']).sum()

# Put node values into array: hidden_layer_outputs

hidden_layer_outputs = np.array([node_0_value, node_1_value])

# Calculate output: output

output = (hidden_layer_outputs*weights['output']).sum()

# Print output

print(output)

The Rectified Linear Activation Function

As Dan explained to you in the video, an "activation function" is a function applied at each node. It converts the node's input into some output.

The rectified linear activation function (called ReLU) has been shown to lead to very high-performance networks. This function takes a single number as an input, returning 0 if the input is negative, and the input if the input is positive.

Here are some examples:
relu(3) = 3
relu(-3) = 0 

Instructions

100 XP

• Fill in the definition of the relu() function:
• Use the max() function to calculate the value for the output of relu().
• Apply the relu() function to node_0_input to calculate node_0_output.
• Apply the relu() function to node_1_input to calculate node_1_output.

Take Hint (-30 XP

def relu(input):

    '''Define your relu activation function here'''

    # Calculate the value for the output of the relu function: output

    output = max(input,0)

   

    # Return the value just calculated

    return(output)

# Calculate node 0 value: node_0_output

node_0_input = (input_data * weights['node_0']).sum()

node_0_output = relu(node_0_input)

# Calculate node 1 value: node_1_output

node_1_input = (input_data * weights['node_1']).sum()

node_1_output = relu(node_1_input)

# Put node values into array: hidden_layer_outputs

hidden_layer_outputs = np.array([node_0_output, node_1_output])

# Calculate model output (do not apply relu)

model_output = (hidden_layer_outputs * weights['output']).sum()

# Print model output

print(model_output) def relu(input):

    '''Define your relu activation function here'''

    # Calculate the value for the output of the relu function: output

    output = max(input,0)

   

    # Return the value just calculated

    return(output)

# Calculate node 0 value: node_0_output

node_0_input = (input_data * weights['node_0']).sum()

node_0_output = relu(node_0_input)

# Calculate node 1 value: node_1_output

node_1_input = (input_data * weights['node_1']).sum()

node_1_output = relu(node_1_input)

# Put node values into array: hidden_layer_outputs

hidden_layer_outputs = np.array([node_0_output, node_1_output])

# Calculate model output (do not apply relu)

model_output = (hidden_layer_outputs * weights['output']).sum()

# Print model output

print(model_output)

**********+

Exercise

Exercise

Applying the network to many observations/rows of data

You'll now define a function called predict_with_network()which will generate predictions for multiple data observations, which are pre-loaded as input_data. As before, weights are also pre-loaded. In addition, the relu() function you defined in the previous exercise has been pre-loaded.

Instructions

0 XP

• Define a function called predict_with_network() that accepts two arguments - input_data_row and weights - and returns a prediction from the network as the output.
• Calculate the input and output values for each node, storing them as: node_0_input, node_0_output, node_1_input, and node_1_output.
• To calculate the input value of a node, multiply the relevant arrays together and compute their sum.
• To calculate the output value of a node, apply the relu()function to the input value of the node.
• Calculate the model output by calculating input_to_final_layer and model_output in the same ay you calculated the input and output values for the nodes.
• Use a for loop to iterate over input_data:
• Use your predict_with_network() to generate predictions for each row of the input_data - input_data_row. Append each prediction to results.

•  

Hint

· To calculate the input value for each node, multiply the two relevant arrays and compute their sum. For example, the two relevant arrays for calculating node_0_input are input_data_row and weights['node_0'].

· To compute the output value of each node, apply the relu()function to the input value.

· Inside the for loop, use the predict_with_network()function with each row of the input data - input_data_row - and weights as the arguments.

Did you find this hint helpful?

# Define predict_with_network()

def predict_with_network(input_data_row, weights):

    # Calculate node 0 value

    node_0_input = (input_data_row * weights['node_0']).sum()

    node_0_output = relu(node_0_input)

    # Calculate node 1 value

    node_1_input = (input_data_row * weights['node_1']).sum()

    node_1_output = relu(node_1_input)

    # Put node values into array: hidden_layer_outputs

    hidden_layer_outputs = np.array([node_0_output, node_1_output])

   

    # Calculate model output

    input_to_final_layer = (hidden_layer_outputs * weights['output']).sum()

    model_output = relu(input_to_final_layer)

   

    # Return model output

    return(model_output)

# Create empty list to store prediction results

results = []

for input_data_row in input_data:

    # Append prediction to results

    results.append(predict_with_network(input_data_row, weights))

# Print results

print(results)

       

def predict_with_network(input_data):

    # Calculate node 0 in the first hidden layer

    node_0_0_input = (input_data *weights['node_0_0']).sum()

    node_0_0_output = relu(node_0_0_input)

    # Calculate node 1 in the first hidden layer

    node_0_1_input = (input_data *weights['node_0_1']).sum()

    node_0_1_output = relu(node_0_1_input)

    # Put node values into array: hidden_0_outputs

    hidden_0_outputs = np.array([node_0_0_output, node_0_1_output])

   

    # Calculate node 0 in the second hidden layer

    node_1_0_input = (input_data *weights['node_1_0']).sum()

    node_1_0_output = relu(node_1_0_input)

    # Calculate node 1 in the second hidden layer

    node_1_1_input = (input_data *weights['node_1_1']).sum()

    node_1_1_output = relu(node_1_1_input)

   

    # Put node values into array: hidden_1_outputs

    hidden_1_outputs = np.array([node_1_0_output, node_1_1_output])

    # Calculate model output: model_output

    model_output = (hidden_1_outputs * weights['output']).sum()

   

    # Return model_output

    return(model_output)

output = predict_with_network(input_data)

print(output)

******************

Correcto , porque no tomeaste en uenta las hidden layers

def predict_with_network(input_data):

    # Calculate node 0 in the first hidden layer

    node_0_0_input = (input_data * weights['node_0_0']).sum()

    node_0_0_output = relu(node_0_0_input)

    # Calculate node 1 in the first hidden layer

    node_0_1_input = (input_data * weights['node_0_1']).sum()

    node_0_1_output = relu(node_0_1_input)

    # Put node values into array: hidden_0_outputs

    hidden_0_outputs = np.array([node_0_0_output, node_0_1_output])

    # Calculate node 0 in the second hidden layer

    node_1_0_input = (hidden_0_outputs * weights['node_1_0']).sum()

    node_1_0_output = relu(node_1_0_input)

    # Calculate node 1 in the second hidden layer

    node_1_1_input = (hidden_0_outputs * weights['node_1_1']).sum()

    node_1_1_output = relu(node_1_1_input)

    # Put node values into array: hidden_1_outputs

    hidden_1_outputs = np.array([node_1_0_output, node_1_1_output])

   

    # Calculate output here: model_output

    model_output = (hidden_1_outputs * weights['output']).sum()

   

    # Return model_output

    return(model_output)

output = predict_with_network(input_data)

print(output)

Multi-layer neural networks

In this exercise, you'll write code to do forward propagation for a neural network with 2 hidden layers. Each hidden layer has two nodes. The input data has been preloaded as input_data. The nodes in the first hidden layer are called node_0_0 and node_0_1. Their weights are pre-loaded as weights['node_0_0'] and weights['node_0_1']respectively.

The nodes in the second hidden layer are called node_1_0 and node_1_1. Their weights are pre-loaded as weights['node_1_0'] and weights['node_1_1']respectively.

We then create a model output from the hidden nodes using weights pre-loaded as weights['output'].

Instructions

0 XP

Instructions

0 XP

• Calculate node_0_0_input using its weights weights['node_0_0'] and the given input_data. Then apply the relu() function to get node_0_0_output.
• Do the same as above for node_0_1_input to get node_0_1_output.
• Calculate node_1_0_input using its weights weights['node_1_0'] and the outputs from the first hidden layer - hidden_0_outputs. Then apply the relu() function to get node_1_0_output.
• Do the same as above for node_1_1_input to get node_1_1_output.
• Calculate model_output using its weights weights['output'] and the outputs from the second hidden layer hidden_1_outputs array. Do not apply the relu()function to this output.

•  

Great link https://jiayiwangjw.github.io/2017/10/03/Introduction-to-Deep-Learning-01/
https://medium.freecodecamp.org/building-a-3-layer-neural-network-from-scratch-99239c4af5d3

Coding how weight changes affect accuracy

Now you'll get to change weights in a real network and see how they affect model accuracy!

Have a look at the following neural network: 

Its weights have been pre-loaded as weights_0. Your task in this exercise is to update a single weight in weights_0 to create weights_1, which gives a perfect prediction (in which the predicted value is equal to target_actual: 3).

Use a pen and paper if necessary to experiment with different combinations. You'll use the predict_with_network() function, which takes an array of data as the first argument, and weights as the second argument.

Instructions

100 XP

• Create a dictionary of weights called weights_1 where you have changed 1 weight from weights_0 (You only need to make 1 edit to weights_0to generate the perfect prediction).
• Obtain predictions with the new weights using the predict_with_network() function with input_data and weights_1.
• Calculate the error for the new weights by subtracting target_actualfrom model_output_1.
• Hit 'Submit Answer' to see how the errors compare!

Take Hint (-30 XP)

# The data point you will make a prediction for
input_data = np.array([0, 3])

# Sample weights
weights_0 = {'node_0': [2, 1],
             'node_1': [1, 2],
             'output': [1, 1]
            }

# The actual target value, used to calculate the error
target_actual = 3

# Make prediction using original weights
model_output_0 = predict_with_network(input_data, weights_0)

# Calculate error: error_0
error_0 = model_output_0 - target_actual

# Create weights that cause the network to make perfect prediction (3): weights_1
weights_1 = {'node_0': [2, 1],
             'node_1': [1, 0],
             'output': [1, 1]
            }

# Make prediction using new weights: model_output_1
model_output_1 = predict_with_network(input_data, weights_1)

# Calculate error: error_1
error_1 = model_output_1 - target_actual

# Print error_0 and error_1
print(error_0)
print(error_1)



RESULT, bingo!

In [2]: # The data point you will make a prediction for
        input_data = np.array([0, 3])
     
        # Sample weights
        weights_0 = {'node_0': [2, 1],
                     'node_1': [1, 2],
                     'output': [1, 1]
                    }
     
        # The actual target value, used to calculate the error
        target_actual = 3
     
        # Make prediction using original weights
        model_output_0 = predict_with_network(input_data, weights_0)
     
        # Calculate error: error_0
        error_0 = model_output_0 - target_actual
     
        # Create weights that cause the network to make perfect prediction (3): weights_1
        weights_1 = {'node_0': [2, 1],
                     'node_1': [1, 0],
                     'output': [1, 1]
                    }
     
        # Make prediction using new weights: model_output_1
        model_output_1 = predict_with_network(input_data, weights_1)
     
        # Calculate error: error_1
        error_1 = model_output_1 - target_actual
     
        # Print error_0 and error_1
        print(error_0)
        print(error_1)
6
0