CS代考程序代写 algorithm Agda scheme python Name: Email: Student ID:

DS100 Midterm A, Page 16 of 27
October 17, 2018
Solution:
def grad(phi, y, theta):
“””Returns dL/dtheta. Assume correct implementation.”””
def grad_desc_momentum(phi, y, num_iter, alpha, gamma, t0): theta, theta_prev = t0, t0
for t in range(num_iter):
g = grad(phi, y, theta)
m = grad(phi, y, theta_prev)
theta, theta_prev = theta – alpha * g – gamma * m,
theta return theta
1 2 3 4 5 6 7 8 9
10 11

DS100 Midterm A, Page 17 of 27 October 17, 2018 13. Consider the following function of f(θ), which alternates between completely flat regions and
regions of absolute slope equal to 1. There is only one correct answer for each part.
(a) Assuming that θ starts in a flat region that is not a minimum and α > 0, will the basic gradient descent algorithm terminate at a minimum? Note that the basic gradient descent algorithm is just the same as version with momentum on the previous page, but where γ = 0.
⃝ A. Never ⃝ B. Maybe ⃝ C. Yes with enough iterations
(b) Assuming that θ starts in a sloped region and α > 0, will the basic gradient descent
algorithm find the minimum?
⃝ A. Never ⃝ B. Maybe ⃝ C. Yes with enough iterations
(c) Assuming that θ starts in a flat region that is not a minimum and α > 0 and γ > 0, will the momentum gradient descent algorithm find the minimum?
⃝ A. Never ⃝ B. Maybe ⃝ C. Yes with enough iterations
(d) Assuming that θ starts in a sloped region and α > 0 and γ > 0, will the momentum
gradient descent algorithm find the minimum?
⃝ A. Never ⃝ B. Maybe ⃝ C. Yes with enough iterations
(e) Is f(θ) convex? ⃝ A. Yes
⃝ B. No
⃝ C. No, but −f(θ) is convex ⃝ D. No, but f(−θ) is convex

DS100 Midterm A, Page 18 of 27 October 17, 2018
Feature Engineering
You Can’t Forget About Those Tips
14. Recallfromlabs5and6thetipsdatasetfromtheseabornlibrary,whichcontainsrecordsabout tips, total bills, and information about the person who paid the tip. There are a total of 244 records in tips. In addition, you can assume that there are no missing or NaN values in the dataset. The first 5 rows of the tips DataFrame are shown below, where sex takes on values ∈ {”M ale”, ”F emale”}, smoker takes on values ∈ {”Y es”, ”N o”}, day takes on values from Monday to Sunday as strings, and time takes on values ∈ {”Breakf ast”, ”Lunch”, ”Dinner”}.
(a) Suppose we use pd.get dummies to create a one-hot encoding of only our sex col- umn. This yields a feature matrix Φq1 with exactly 2 columns sex Male, sex Female, where values can be either 0 or 1 in each column.
Which of the following are true? Select all that apply. 􏰄 A. Φq1 has 244 rows.
􏰄 B. Φq1 has full column rank.
􏰄 C. (ΦTq1Φq1) is invertible.
􏰄 D. None of the above
Solution:
􏰄 A. True. The resulting matrix has one row for very row in the original tips table.
􏰄 B. True. The two columns are linearly independent. 􏰄 C. True, because Φq1 has full column rank.
􏰄 D. False. The above answer choices are true.
(b) Suppose we use pd.get dummies to create a one-hot encoding of only our sex and smoker columns. This yields a feature matrix Φq2 with 4 columns.

DS100
Midterm A, Page 19 of 27
October 17, 2018
Which of the following are true? Select all that apply. 􏰄 A. Φq2 has 244 rows.
􏰄 B. Φq2 has full column rank.
􏰄 C. (ΦTq2Φq2) is invertible.
􏰄 D. None of the above
Solution:
􏰄 A. 􏰄 B. 􏰄 C. 􏰄 D.
True. The resulting matrix has one row for very row in the original tips table.
False. We can generate the sex male column by adding our smoker columns and then subtracting the female column.
False. Since Φq2 doesn’t have full column rank, this means that (ΦTq2Φq2) won’t be invertible.
False. We chose an answer choice above.

DS100 Midterm A, Page 20 of 27 October 17, 2018
(c) Suppose we use pd.get dummies to create a one-hot encoding of only our sex and smoker columns, and also include a bias column. This yields a feature matrix Φq3 with 5 columns.
Which of the following are true? Select all that apply. 􏰄 A. Φq3 has 244 rows.
􏰄 B. Φq3 has full column rank.
􏰄 C. (ΦTq3Φq3) is invertible.
􏰄 D. None of the above
Solution:
􏰄 A.
􏰄 B. 􏰄 C. 􏰄 D.
True. The resulting matrix has one row for very row in the original tips table.
False. Same as the previous problem. False. Same as the previous problem. False. We chose an answer choice above.
(d) For the day column, we can either use a one-hot encoding or an integer encoding. By integer encoding, we mean mapping Monday to 1, Tuesday to 2, and so on. Which of the following statements are true? Select all that apply.
􏰄 A. 􏰄 B.
􏰄 C. 􏰄 D.
One-hot encoding creates fewer columns than integer encoding.
One-hot encoding gives all days of the week the same weight, while integer encoding gives certain days of the week higher weight than others.
The columns generated by the one-hot encoding of the days of the week are linearly independent of each other.
None of the above
Solution:
􏰄 A.
􏰄 B. 􏰄 C.
False. One-hot encoding creates more columns than integer encoding, as a column is added for every non-numerical value a certain feature can take on.
True. This is exactly why we one-hot encode non-numerical values.
True. The columns generated are linearly independent of each other (but may not be when combined with other columns, e.g. a bias col- umn).
􏰄 D. False. We chose 2 of the answer choices above.

DS100 Midterm A, Page 21 of 27 October 17, 2018
More Feature Engineering
15. Which of the following are reasons to use N -Gram Encoding (N > 1) instead of Bag-of-words (1-Gram) encoding? For simplicity, you may assume that N = 2 for N-Gram Encoding. Select all that apply.
􏰄 A. 􏰄 B. 􏰄 C.
Vectors for N -Gram encoding are less sparse than vectors from Bag-of-words en- coding.
Vectors for N -Gram encoding have lower dimension than vectors from Bag-of- words encoding.
It is easier for N -Gram encoding to deal with unseen combinations at prediction time.
􏰄 D. N -Gram helps preserve word order while Bag-of-words does not. 􏰄 E. None of the above
Solution:
􏰄 A. 􏰄 B.
False. Vectors for N -Gram are usually more sparse than Bag-of-words vectors. False. Vectors for N -Gram usually have higher dimension than Bag-of-words
vectors.
􏰄 C.
􏰄 D. True. N -gram does help preserve word order while Bag-of-words does not.
False. Both encoding schemes have to deal with the issue of unseen combina- tions at prediction time.

DS100 Midterm A, Page 22 of 27 October 17, 2018
Regularization
Elastic Net Regularization
16. Elastic Net is a regression technique that combines L1 and L2 regularization. It is preferred in many situations as it possesses the benefits of both LASSO and Ridge Regression. Minimizing theL2lossusingElasticNetisasfollows,whereλ1,λ2 >=0,λ1 +λ2 =λ,λ>0.
1pp θˆ=argmin 􏰃(y −θx)2 +λ 􏰃|θ |+λ 􏰃θ2
θni1j2j i j=1 j=1
Suppose our goal was to get sparse parameters, i.e. we want as many parameters as possible to be zero. Which of the following choices for λ1, λ2 are most consistent with this goal, assuming λ = 1? There is only one correct answer.
⃝A. λ1 =0,λ2 =1 ⃝B. λ1 =0.5,λ2 =0.5 ⃝C. λ1 =1,λ2 =0
17. What happens to bias and variance as we increase the value of λ? Assume λ2 = λ1. There is only one correct answer in each part. You will be asked to justify why in the next question.
Solution: We know that LASSO encourages sparsity in our optimal weights. Setting λ1 to 1 means we are using LASSO.
(a) Bias:
⃝ A. ⃝ B. ⃝ C.
(b) Variance: ⃝ A. ⃝ B. ⃝ C.
Bias goes up
Bias stays the same Bias goes down
Variance goes up Variance stays the same Variance goes down
18. Justify why by marking the true statements. Select all that apply for each part.
(a) Bias:
􏰄 A. Bias goes down because increasing λ reduces over fitting. 􏰄 B. Bias goes down because bias is minimized when λ2 = λ1.

DS100
Midterm A, Page 23 of 27 October 17, 2018
􏰄 C.
􏰄 D.
􏰄 E. (b) Variance:
􏰄 A.
􏰄 B. 􏰄 C. 􏰄 D.
􏰄 E. 19. What happens
apply.
Bias goes up because increasing λ penalizes complex models, limiting the set of possible solutions.
Bias goes up because the loss function becomes non-convex for sufficiently large λ.
None of the above
Variance goes down because increasing λ encourages the value of the loss to decrease.
Variance goes down because increasing λ penalizes large model weights. Variance goes up because because increasing λ increases bias.
Variance goes up because increasing λ increases the magnitude of terms in the loss function.
None of the above
to the model parameters θ as λ → ∞, i.e. what is limλ→∞ θ? Select all that
􏰄 A. Converge to 0.
􏰄 B. Diverge to infinity.
􏰄 C. Converge to values that minimize the L2 loss. 􏰄 D. Converge to equal but non-zero values.
􏰄 E. Converge to a sparse vector.
ˆˆ
Solution: The model parameters go to 0.
20. Of the choices below, why do we prefer to use ridge regression over linear regression (i.e. the normal equation) in certain cases? Select all that apply.
􏰄 A. 􏰄 B. 􏰄 C. 􏰄 D.
Ridge regression always guarantees an analytic solution, but the normal equa- tion does not.
Ridge regression encourages sparsity in our model parameters, which is helpful for inferring useful features.
Ridge regression isn’t sensitive to outliers, which makes it preferable over linear regression.
Ridge regression always performs just as well as linear regression, with the added benefit of reduced variance.

DS100
Midterm A, Page 24 of 27 October 17, 2018
􏰄 E. None of the above
Solution:
􏰄 A.
􏰄 B. 􏰄 C. 􏰄 D.
The regularization term guarantees (AT A + λI) is invertible, as discussed in discussion 7.
This is the description for LASSO. It is sensitive to outliers.
Doesn’t always perform better.

DS100 Midterm A, Page 25 of 27 October 17, 2018
More Linear Models
21. Suppose in some universe, the true relationship between the measured luminosity of a single star Y can be written in terms of a single feature φ of that same star as
Y =θ∗φ+ε
where φ ∈ R is some non-random scalar feature, θ∗ ∈ R is a non-random scalar parameter,
and ε is a random variable with E[ε] = 0 and Var(ε) = σ2. For each star, you have a set
of features Φ = 􏰈φ1 φ2 … φn􏰉T and luminosity measurements y = 􏰈y1 y2 … yn􏰉T generated by this relationship. Your Φ may or may not include the feature φ described above. The εi for the various yi have the same probability distribution and are independent of each other.
(a) What is E[Y ]?
⃝A. 0 ⃝B. θ∗φ ⃝C. φ(ΦTΦ)−1ΦTy
⃝D. θ∗ ⃝E. Noneoftheabove
Solution: E[Y]=E[θ∗φ+ε]=θ∗φ+0
(b) What is Var(Y )?
σ2 σ2
⃝A.n ⃝B.n2 ⃝C.0
1 􏰂n 􏰀 1 􏰂n 􏰁 2
⃝D. n−1 yi−n yi ⃝E. Noneoftheabove
i=1 i=1
(c) Suppose you have information about the exact φ value for each star, but try to fit a linear model for Y that includes an intercept term θ0.
Y = θ0 + θ1φ
Note the true relationship has no intercept term, so our model is not quite correct. Let θˆ
and θˆ be the values that minimize the average L loss. Let y be the actual observed data 12
and yˆ = θˆ + θˆ φ be the fitted values. 01
i. Which of the following could possibly be the value of θˆ after fitting our model? 0
Select all that apply; at least one is correct.
􏰄A.-1 􏰄B.0 􏰄C.1 􏰄D.10
Solution: Var(Y ) = Var(θ∗x + ε) = Var(ε) = σ2
0

DS100 Midterm A, Page 26 of 27 October 17, 2018
Solution: There are no restrictions on θˆ given our assumptions. 0
ii. Which of the following could possibly be the residual vector for our model? Select all that apply; at least one is correct.
􏰄 A. 􏰈−2 −4 6􏰉T 􏰄 B. 􏰈0.0001 0.0003 −0.0005􏰉T 􏰄C. 􏰈3 12 −9􏰉T 􏰄D. 􏰈1 1 1􏰉T
Solution: Since we are including an intercept/bias term, Φ has a column of 1s, which we denote with a boldface 1. Optimality requires orthogonality of the residual vector with the column space of Φ, which requires 1T e = 􏰂ni=1 1×ei = 􏰂ni=1 ei = 0. (A) is the only choice satisfying this condition.

DS100 Midterm A, Page 27 of 27 October 17, 2018
22. Suppose we create a new loss function called the OINK loss, defined as follows for a single observation:
􏰒a(fθ(x) − y) fθ(x) ≥ y LOINK(θ,x,y)= b(y−fθ(x)) fθ(x)