tag:blogger.com,1999:blog-7958828565254404797.post6670273742051793277..comments2024-03-28T07:44:59.527-07:00Comments on ListenData: When and why to standardize a variableDeepanshu Bhallahttp://www.blogger.com/profile/09802839558125192674noreply@blogger.comBlogger13125tag:blogger.com,1999:blog-7958828565254404797.post-45811418344312285552022-01-01T20:10:25.899-08:002022-01-01T20:10:25.899-08:00Thank you very much for your blog. Your blog is th...Thank you very much for your blog. Your blog is the most comprehensive and detailed explanation of why scaling is done among the resources I can find on the Internet.Anonymoushttps://www.blogger.com/profile/17313305956666888935noreply@blogger.comtag:blogger.com,1999:blog-7958828565254404797.post-44680671648896926652021-10-07T10:58:00.577-07:002021-10-07T10:58:00.577-07:00Range cannot be zero. A variable with zero range i...Range cannot be zero. A variable with zero range is a constant. Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-7958828565254404797.post-91228762416808791002021-09-30T07:24:42.105-07:002021-09-30T07:24:42.105-07:00What if the range is zero for some observations? I...What if the range is zero for some observations? In the case of the range method for example, the divisor would be zero for these observations. How can we accommodate for this?Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-7958828565254404797.post-13524996807816950092021-04-09T00:49:56.754-07:002021-04-09T00:49:56.754-07:00Does it matter if the variables that you are scali...Does it matter if the variables that you are scaling are normally distributed or not?Anonymoushttps://www.blogger.com/profile/16736440537646754471noreply@blogger.comtag:blogger.com,1999:blog-7958828565254404797.post-3826547959202324882021-02-25T12:10:49.035-08:002021-02-25T12:10:49.035-08:00Normalization won't improve model performance,...Normalization won't improve model performance, but it will affect the values of MAE, MSE and RMSE, but not MAPE. See below example of the metrics from the same model with the same observed and predicted values, but with results in dollars (left) and pesos (right). This is the scalability problem of the metrics.<br />MAE 0,383 47,048<br />MSE 0,247 3741,780<br />RMSE 0,497 61,170<br />MAPE 1,33 1,33<br />Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-7958828565254404797.post-54083203763768701662020-12-22T09:48:50.573-08:002020-12-22T09:48:50.573-08:00Congratulations on your comprehensive and easy-rea...Congratulations on your comprehensive and easy-readable post on something so important for any modern data engineer. I would like to add from personal experience that certain monotonic functions(e.g. cubic root) can be used after subtracting each variable's mean over the used sample (always after error correction and imputation) in a linear regression in order to: a) efficiently scale well any outliers b) efficiently compare any measures of different scales <br />c) linearly & possibly non-linearly detrend your variable (needed for stationarity assumptions in time series models)<br />Keep up the good work!wsssshttps://www.blogger.com/profile/17169170533860985037noreply@blogger.comtag:blogger.com,1999:blog-7958828565254404797.post-71576169401435027342020-12-22T09:45:39.965-08:002020-12-22T09:45:39.965-08:00Congratulations on your comprehensive and easy-rea...Congratulations on your comprehensive and easy-readable post on something so important for any modern data engineer. I would like to add from personal experience that certain monotonic functions(e.g. cubic root) can be used after subtracting each variable's mean over the used sample (always after error correction and imputation) in a linear regression in order to: a) efficiently scale well any outliers b) efficiently compare any measures of different scales <br />c) linearly & possibly non-linearly detrend your variable (needed for stationarity assumptions in time series models)<br />Keep up the good work!wsssshttps://www.blogger.com/profile/17169170533860985037noreply@blogger.comtag:blogger.com,1999:blog-7958828565254404797.post-85771444701435555632020-11-24T12:11:52.733-08:002020-11-24T12:11:52.733-08:00What is the need of performing a standardization.I...What is the need of performing a standardization.If is not a implementing our model performance. Prashanthhttps://www.blogger.com/profile/01491232859544987126noreply@blogger.comtag:blogger.com,1999:blog-7958828565254404797.post-12043927397712092852019-05-27T23:12:51.844-07:002019-05-27T23:12:51.844-07:00I am novice in Data Science, Could you please also...I am novice in Data Science, Could you please also mention packages which needs to import for doing these calculations.Nitin4Uhttps://www.blogger.com/profile/13643144663611969847noreply@blogger.comtag:blogger.com,1999:blog-7958828565254404797.post-11188196254935890562018-08-28T14:37:21.646-07:002018-08-28T14:37:21.646-07:00how do one standardize variables when the feature ...how do one standardize variables when the feature variables have different data types, can we go with one method for each feature and still try out different methods on different features, is that a correct option or <br /><br />a) use only one method of standardization in a case where different data types are available as part of standardization- say centring-- by subtracting the means - but what if the feature is categorical- can we subtract mode instead or should we follow a common procedure Anonymoushttps://www.blogger.com/profile/04821212094141312359noreply@blogger.comtag:blogger.com,1999:blog-7958828565254404797.post-63408497313561013972018-05-29T04:38:21.037-07:002018-05-29T04:38:21.037-07:00This is one of the best tutorial. Thanks for the g...This is one of the best tutorial. Thanks for the great effort.Nageshttps://www.blogger.com/profile/01369203023406077033noreply@blogger.comtag:blogger.com,1999:blog-7958828565254404797.post-22436813990431713652017-04-14T07:56:55.154-07:002017-04-14T07:56:55.154-07:00There is no thumb rule regarding the standardizati...There is no thumb rule regarding the standardization method. It depends on your dataset. You need to apply methods and see which method works for you. Thanks!Deepanshu Bhallahttps://www.blogger.com/profile/09802839558125192674noreply@blogger.comtag:blogger.com,1999:blog-7958828565254404797.post-80095806908400763282017-04-12T07:02:21.972-07:002017-04-12T07:02:21.972-07:00Useful article. Although you did a good job explai...Useful article. Although you did a good job explaining why and when you might want to standardize a variable, you don't mention what criteria to use for actually selecting a standardizing method. For example, under what circumstances would it be better to use Z score vs. Min/Max?Unknownhttps://www.blogger.com/profile/17042730599217214919noreply@blogger.com