{"id":23185,"date":"2022-11-12T15:34:56","date_gmt":"2022-11-12T15:34:56","guid":{"rendered":"https:\/\/khumbuphotography.com\/?p=23185"},"modified":"2022-11-12T15:41:07","modified_gmt":"2022-11-12T15:41:07","slug":"first-note-that-the-smallest-l2-norm-vector-that-6","status":"publish","type":"post","link":"https:\/\/khumbuphotography.com\/?p=23185","title":{"rendered":"First, note that the smallest L2-norm vector that can fit the training data for the core model is \\(>=[2,0,0]\\)"},"content":{"rendered":"<p><title>First, note that the smallest L2-norm vector that can fit the training data for the core model is \\(<\\theta^\\text<-s>>=[2,0,0]\\)<\/title><\/p>\n<p>On the other hand, in the presence of the spurious feature, the full model can fit the training data perfectly with a smaller norm by assigning weight \\(1\\) for the feature \\(s\\) (\\(|<\\theta^\\text<-s>>|_2^2 = 4\\) while \\(|<\\theta^\\text<+s>>|_2^2 + w^2 = 2 < 4\\)).<\/p>\n<p>Generally, in the overparameterized regime, since the number of training examples is less than the number of features, there are some directions of data variation that are not observed in the training data. In this example, we do not observe any information about the second and third features. However, the non-zero weight for the spurious feature leads to a different assumption for the unseen directions. In particular, the full model does not assign weight \\(0\\) to the unseen directions. Indeed, by substituting \\(s\\) with \\(<\\beta^\\star>^\\top z\\), we can view the full model as not using \\(s\\) but implicitly assigning weight \\(\\beta^\\star_2=2\\) to the second feature and \\(\\beta^\\star_3=-2\\) to the third feature (unseen directions at training).<\/p>\n<p>In this analogy, removing \\(s\\) decreases the mistake having an examination delivery with a high deviations of zero with the second function, whereas removing \\(s\\) boosts the error getting a test shipments with high deviations regarding zero toward third element.<\/p>\n<h2>Drop <a href=\"https:\/\/datingranking.net\/escort-directory\/vancouver\/\">Vancouver WA eros escort<\/a> in accuracy in test time depends on the relationship between the true target parameter (\\(\\theta^\\star\\)) and the true spurious feature parameters (\\(<\\beta^\\star>\\)) in the seen directions and unseen direction<\/h2>\n<p>As we saw in the previous example, by using the spurious feature, the full model incorporates \\(<\\beta^\\star>\\) into its estimate. The true target parameter (\\(\\theta^\\star\\)) and the true spurious feature parameters (\\(<\\beta^\\star>\\)) agree on some of the unseen directions and do not agree on the others. Thus, depending on which unseen directions are weighted heavily in the test time, removing \\(s\\) can increase or decrease the error.<!--more--><\/p>\n<p>More formally, the weight assigned to the spurious feature is proportional to the projection of \\(\\theta^\\star\\) on \\(<\\beta^\\star>\\) on the seen directions. If this number is close to the projection of \\(\\theta^\\star\\) on \\(<\\beta^\\star>\\) on the unseen directions (in comparison to 0), removing \\(s\\) increases the error, and it decreases the error otherwise. Note that since we are assuming noiseless linear regression and choose models that fit training data, the model predicts perfectly in the seen directions and only variations in unseen directions contribute to the error.<\/p>\n<p>(Left) New projection off \\(\\theta^\\star\\) on \\(\\beta^\\star\\) was self-confident from the viewed assistance, however it is negative about unseen direction; therefore, removing \\(s\\) reduces the error. (Right) The new projection out-of \\(\\theta^\\star\\) toward \\(\\beta^\\star\\) is similar in both seen and you will unseen guidelines; ergo, removing \\(s\\) escalates the error.<\/p>\n<p>Let&#8217;s now formalize the conditions under which removing the spurious feature (\\(s\\)) increases the error. Let \\(\\Pi = Z(ZZ^\\top)^<-1>Z\\) denote the column space of training data (seen directions), thus \\(I-\\Pi\\) denotes the null space of training data (unseen direction). The below equation determines when removing the spurious feature decreases the error.<\/p>\n<h2>The new center design assigns lbs \\(0\\) to your unseen instructions (pounds \\(0\\) into the next and you may 3rd features inside example)<\/h2>\n<p>The newest kept side &#8216;s the difference between this new projection off \\(\\theta^\\star\\) towards the \\(\\beta^\\star\\) throughout the viewed guidance using their projection from the unseen guidelines scaled because of the decide to try date covariance. Best top &#8216;s the difference in 0 (i.elizabeth., not using spurious has actually) and the projection of \\(\\theta^\\star\\) on \\(\\beta^\\star\\) in the unseen assistance scaled of the sample time covariance. Deleting \\(s\\) facilitate in the event your remaining front side is actually higher than suitable front.<\/p>\n<p>As principle can be applied simply to linear activities, we now show that within the low-linear designs trained with the actual-business datasets, deleting an excellent spurious feature decreases the accuracy and influences organizations disproportionately.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>First, note that the smallest L2-norm vector that can fit the training data for the core model is \\(=[2,0,0]\\) On the other hand, in the&hellip; <\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[8758],"tags":[],"class_list":["post-23185","post","type-post","status-publish","format-standard","hentry","category-vancouver-escort-2"],"_links":{"self":[{"href":"https:\/\/khumbuphotography.com\/index.php?rest_route=\/wp\/v2\/posts\/23185"}],"collection":[{"href":"https:\/\/khumbuphotography.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/khumbuphotography.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/khumbuphotography.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/khumbuphotography.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=23185"}],"version-history":[{"count":1,"href":"https:\/\/khumbuphotography.com\/index.php?rest_route=\/wp\/v2\/posts\/23185\/revisions"}],"predecessor-version":[{"id":23186,"href":"https:\/\/khumbuphotography.com\/index.php?rest_route=\/wp\/v2\/posts\/23185\/revisions\/23186"}],"wp:attachment":[{"href":"https:\/\/khumbuphotography.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=23185"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/khumbuphotography.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=23185"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/khumbuphotography.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=23185"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}