# [cs231n] assignment1 SVM分类器部分

1. 完成一个使用向量方法计算svm损失函数;
2. 完成一个使用向量方法来分析梯度;
3. 使用数学方法来检查梯度
4. 用验证集来微调学习率和正则项;
5. 使用随机梯度下降法来优化损失函数; 6. 可视化最后学习到的权重

SVM分类器的损失函数的计算公式如下：

$$L = \frac{1}{N}\sum_{i}\sum_{j\neq y_i}[max(0, f_j(x_i; W)-f_{y_i}(x_i;W)+\triangle)] + \lambda \sum_k \sum_lW_{k,l}^2$$

$$\frac{df(x)}{dx} = \lim\limits_{h\to0} \frac{f(x+h)-f(x)}{h}$$

$$\nabla{L_i} = -(\sum_{j\neq{y_i}}1(w_j^Tx_i-w_{y_i}^T + \triangle > 0)) x_i$$

## svm.ipvnb

Complete and hand in this completed worksheet (including its outputs and any supporting code outside of the worksheet) with your assignment submission. For more details see the assignments page on the course website.

In this exercise you will:

• implement a fully-vectorized loss function for the SVM
• implement the fully-vectorized expression for its analytic gradient
• use a validation set to tune the learning rate and regularization strength
• optimize the loss function with SGD
• visualize the final learned weights

## SVM Classifier

Your code for this section will all be written inside cs231n/classifiers/linear_svm.py.

As you can see, we have prefilled the function compute_loss_naive which uses for loops to evaluate the multiclass SVM loss function.

The grad returned from the function above is right now all zero. Derive and implement the gradient for the SVM cost function and implement it inline inside the function svm_loss_naive. You will find it helpful to interleave your new code inside the existing function.

To check that you have correctly implemented the gradient correctly, you can numerically estimate the gradient of the loss function and compare the numeric estimate to the gradient that you computed. We have provided code that does this for you:

### Inline Question 1:

It is possible that once in a while a dimension in the gradcheck will not match exactly. What could such a discrepancy be caused by? Is it a reason for concern? What is a simple example in one dimension where a gradient check could fail? How would change the margin affect of the frequency of this happening? Hint: the SVM loss function is not strictly speaking differentiable

We now have vectorized and efficient expressions for the loss, the gradient and our gradient matches the numerical gradient. We are therefore ready to do SGD to minimize the loss.

### Inline question 2:

Describe what your visualized SVM weights look like, and offer a brief explanation for why they look they way that they do.