📚 Accelerators for Convolutional Neural Networks (2023)

1⃣ Join Channel Download:
https://yangx.top/+MhmkscCzIYQ2MmM8

2⃣ Download Book: https://yangx.top/c/1854405158/768

💬 Tags: #cnn

👉 BEST DATA SCIENCE CHANNELS ON TELEGRAM 👈

Topic: CNN (Convolutional Neural Networks) – Part 1: Introduction and Basic Concepts

---

1. What is a CNN?

• A Convolutional Neural Network (CNN) is a type of deep learning model primarily used for analyzing visual data.

• CNNs automatically learn spatial hierarchies of features through convolutional layers.

---

2. Key Components of CNN

• Convolutional Layer: Applies filters (kernels) to input images to extract features like edges, textures, and shapes.

• Activation Function: Usually ReLU (Rectified Linear Unit) is applied after convolution for non-linearity.

• Pooling Layer: Reduces the spatial size of feature maps, typically using Max Pooling.

• Fully Connected Layer: After feature extraction, maps features to output classes.

---

3. How Convolution Works

• A kernel (small matrix) slides over the input image, computing element-wise multiplications and summing them up to form a feature map.

• Kernels detect features like edges, lines, and patterns.

---

4. Basic CNN Architecture Example

| Layer Type | Description |
| --------------- | ---------------------------------- |
| Input | Image of size (e.g., 28x28x1) |
| Conv Layer | 32 filters of size 3x3 |
| Activation | ReLU |
| Pooling Layer | MaxPooling 2x2 |
| Fully Connected | Flatten + Dense for classification |

---

5. Simple CNN with PyTorch Example

import torch.nn as nn
import torch.nn.functional as F

class SimpleCNN(nn.Module):
    def __init__(self):
        super(SimpleCNN, self).__init__()
        self.conv1 = nn.Conv2d(1, 32, kernel_size=3)  # 1 input channel, 32 filters
        self.pool = nn.MaxPool2d(2, 2)
        self.fc1 = nn.Linear(32 * 13 * 13, 10)  # Assuming input 28x28

    def forward(self, x):
        x = self.pool(F.relu(self.conv1(x)))
        x = x.view(-1, 32 * 13 * 13)  # Flatten
        x = self.fc1(x)
        return x

---

6. Why CNN over Fully Connected Networks?

• CNNs reduce the number of parameters by weight sharing in kernels.

• They preserve spatial relationships unlike fully connected layers.

---

Summary

• CNNs are powerful for image and video tasks due to convolution and pooling.

• Understanding convolution, pooling, and architecture basics is key to building models.

---

Exercise

• Implement a CNN with two convolutional layers and train it on MNIST digits.

---

#CNN #DeepLearning #NeuralNetworks #Convolution #MachineLearning

https://yangx.top/DataScience4

Topic: CNN (Convolutional Neural Networks) – Part 2: Layers, Padding, Stride, and Activation Functions

---

1. Convolutional Layer Parameters

• Kernel (Filter) Size: Size of the sliding window (e.g., 3x3, 5x5).

• Stride: Number of pixels the filter moves at each step. Larger stride means smaller output.

• Padding: Adding zeros around the input to control output size.

* Valid padding: No padding, output smaller than input.

* Same padding: Pads input so output size equals input size.

---

2. Calculating Output Size

For input size $N$, filter size $F$, padding $P$, stride $S$:

$$
\text{Output size} = \left\lfloor \frac{N - F + 2P}{S} \right\rfloor + 1
$$

---

3. Activation Functions

• ReLU (Rectified Linear Unit): Most common, outputs zero for negatives, linear for positives.

• Other activations: Sigmoid, Tanh, Leaky ReLU.

---

4. Pooling Layers

• Reduces spatial dimensions to lower computational cost.

• Max Pooling: Takes the maximum value in a window.

• Average Pooling: Takes the average value.

---

5. Example PyTorch CNN with Padding and Stride

import torch.nn as nn
import torch.nn.functional as F

class CNNWithPadding(nn.Module):
    def __init__(self):
        super(CNNWithPadding, self).__init__()
        self.conv1 = nn.Conv2d(1, 16, kernel_size=3, stride=1, padding=1)  # output same size as input
        self.pool = nn.MaxPool2d(2, 2)
        self.conv2 = nn.Conv2d(16, 32, kernel_size=3, stride=1, padding=0)  # valid padding
        self.fc1 = nn.Linear(32 * 13 * 13, 10)

    def forward(self, x):
        x = self.pool(F.relu(self.conv1(x)))  # 28x28 -> 28x28 -> 14x14 after pooling
        x = F.relu(self.conv2(x))              # 14x14 -> 12x12
        x = x.view(-1, 32 * 12 * 12)
        x = self.fc1(x)
        return x

---

6. Summary

• Padding and stride control output dimensions of convolution layers.

• ReLU is widely used for non-linearity.

• Pooling layers reduce dimensionality, improving performance.

---

Exercise

• Modify the example above to add a third convolutional layer with stride 2 and observe output sizes.

---

#CNN #DeepLearning #ActivationFunctions #Padding #Stride

https://yangx.top/DataScience4

Topic: CNN (Convolutional Neural Networks) – Part 3: Batch Normalization, Dropout, and Regularization

---

1. Batch Normalization (BatchNorm)

• Normalizes layer inputs to improve training speed and stability.

• It reduces internal covariate shift by normalizing activations over the batch.

• Formula applied for each batch:

$$
\hat{x} = \frac{x - \mu}{\sqrt{\sigma^2 + \epsilon}} \quad;\quad y = \gamma \hat{x} + \beta
$$

where $\mu$, $\sigma^2$ are batch mean and variance, $\gamma$ and $\beta$ are learnable parameters.

---

2. Dropout

• A regularization technique that randomly "drops out" neurons during training to prevent overfitting.

• The dropout rate (e.g., 0.5) specifies the probability of dropping a neuron.

---

3. Adding BatchNorm and Dropout in PyTorch

import torch.nn as nn
import torch.nn.functional as F

class CNNWithBNDropout(nn.Module):
    def __init__(self):
        super(CNNWithBNDropout, self).__init__()
        self.conv1 = nn.Conv2d(1, 32, 3, padding=1)
        self.bn1 = nn.BatchNorm2d(32)
        self.dropout = nn.Dropout(0.5)
        self.pool = nn.MaxPool2d(2, 2)
        self.fc1 = nn.Linear(32 * 14 * 14, 128)
        self.fc2 = nn.Linear(128, 10)

    def forward(self, x):
        x = self.pool(F.relu(self.bn1(self.conv1(x))))
        x = x.view(-1, 32 * 14 * 14)
        x = F.relu(self.fc1(x))
        x = self.dropout(x)
        x = self.fc2(x)
        return x

---

4. Why Use BatchNorm and Dropout?

• BatchNorm helps the model converge faster and allows higher learning rates.

• Dropout helps reduce overfitting by making the network less sensitive to specific neuron weights.

---

5. Other Regularization Techniques

• Weight Decay: Adds an L2 penalty to weights during optimization.

• Early Stopping: Stops training when validation loss starts increasing.

---

Summary

• Batch normalization and dropout are essential tools for training deep CNNs effectively.

• Regularization improves generalization and reduces overfitting.

---

Exercise

• Modify the CNN above by adding dropout after the second fully connected layer and train it on a dataset to compare results with/without dropout.

---

#CNN #BatchNormalization #Dropout #Regularization #DeepLearning

https://yangx.top/DataScienceM

Topic: CNN (Convolutional Neural Networks) – Part 3: Flattening, Fully Connected Layers, and Final Output

---

1. Flattening the Feature Maps

• After convolution and pooling layers, the resulting feature maps are multi-dimensional tensors.

• Flattening transforms these 3D tensors into 1D vectors to be passed into fully connected (dense) layers.

Example:

x = x.view(x.size(0), -1)

This reshapes the tensor from shape [batch_size, channels, height, width] to [batch_size, features].

---

2. Fully Connected (Dense) Layers

• These layers are used to perform classification based on the extracted features.

• Each neuron is connected to every neuron in the previous layer.

• They are placed after convolutional and pooling layers.

---

3. Output Layer

• The final layer is typically a fully connected layer with output neurons equal to the number of classes.

• Apply a softmax activation for multi-class classification (e.g., 10 classes for digits 0–9).

---

4. Complete CNN Example (PyTorch)

import torch.nn as nn
import torch.nn.functional as F

class FullCNN(nn.Module):
    def __init__(self):
        super(FullCNN, self).__init__()
        self.conv1 = nn.Conv2d(1, 32, 3, padding=1)
        self.pool = nn.MaxPool2d(2, 2)
        self.conv2 = nn.Conv2d(32, 64, 3, padding=1)
        self.fc1 = nn.Linear(64 * 7 * 7, 128)  # assumes input 28x28
        self.fc2 = nn.Linear(128, 10)

    def forward(self, x):
        x = self.pool(F.relu(self.conv1(x)))   # 28x28 -> 14x14
        x = self.pool(F.relu(self.conv2(x)))   # 14x14 -> 7x7
        x = x.view(-1, 64 * 7 * 7)             # Flatten
        x = F.relu(self.fc1(x))
        x = self.fc2(x)                        # Output layer
        return x

---

5. Why Fully Connected Layers Are Important

• They combine all learned spatial features into a single feature vector for classification.

• They introduce the final decision boundary between classes.

---

Summary

• Flattening bridges the convolutional part of the network to the fully connected part.

• Fully connected layers transform features into class scores.

• The output layer applies classification logic like softmax or sigmoid depending on the task.

---

Exercise

• Modify the CNN above to classify CIFAR-10 images (3 channels, 32x32) and calculate the total number of parameters in each layer.

---

#CNN #NeuralNetworks #Flattening #FullyConnected #DeepLearning

https://yangx.top/DataScienceM

Topic: CNN (Convolutional Neural Networks) – Part 4: Training, Loss Functions, and Evaluation Metrics

---

1. Preparing for Training

To train a CNN, we need:

• Dataset – Typically image data with labels (e.g., MNIST, CIFAR-10).

• Loss Function – Measures the difference between predicted and actual values.

• Optimizer – Updates model weights based on gradients.

• Evaluation Metrics – Accuracy, precision, recall, F1 score, etc.

---

2. Common Loss Functions for CNNs

• CrossEntropyLoss – For multi-class classification (most common).

criterion = nn.CrossEntropyLoss()

• BCELoss – For binary classification.

---

3. Optimizers

• SGD (Stochastic Gradient Descent)
• Adam – Adaptive learning rate; widely used for faster convergence.

optimizer = torch.optim.Adam(model.parameters(), lr=0.001)

---

4. Basic Training Loop in PyTorch

for epoch in range(num_epochs):
    model.train()
    running_loss = 0.0

    for images, labels in train_loader:
        optimizer.zero_grad()
        outputs = model(images)
        loss = criterion(outputs, labels)
        loss.backward()
        optimizer.step()
        running_loss += loss.item()

    print(f"Epoch {epoch+1}, Loss: {running_loss:.4f}")

---

5. Evaluating the Model

correct = 0
total = 0
model.eval()

with torch.no_grad():
    for images, labels in test_loader:
        outputs = model(images)
        _, predicted = torch.max(outputs, 1)
        total += labels.size(0)
        correct += (predicted == labels).sum().item()

accuracy = 100 * correct / total
print(f"Test Accuracy: {accuracy:.2f}%")

---

6. Tips for Better CNN Training

• Normalize images.

• Shuffle training data for better generalization.

• Use validation sets to monitor overfitting.

• Save checkpoints (torch.save(model.state_dict())).

---

Summary

• CNN training involves feeding batches of images, computing loss, backpropagation, and updating weights.

• Evaluation metrics like accuracy help track progress.

• Loss functions and optimizers are critical for learning quality.

---

Exercise

• Train a CNN on CIFAR-10 for 10 epochs using CrossEntropyLoss and Adam, then print accuracy and plot loss over epochs.

---

#CNN #DeepLearning #Training #LossFunction #ModelEvaluation

https://yangx.top/DataScienceM