Notice: Function _load_textdomain_just_in_time was called incorrectly. Translation loading for the astra-sites domain was triggered too early. This is usually an indicator for some code in the plugin or theme running too early. Translations should be loaded at the init action or later. Please see Debugging in WordPress for more information. (This message was added in version 6.7.0.) in /home/statplace/public_html/site/wp-includes/functions.php on line 6114

Notice: Function _load_textdomain_just_in_time was called incorrectly. Translation loading for the jetpack domain was triggered too early. This is usually an indicator for some code in the plugin or theme running too early. Translations should be loaded at the init action or later. Please see Debugging in WordPress for more information. (This message was added in version 6.7.0.) in /home/statplace/public_html/site/wp-includes/functions.php on line 6114

Notice: Function _load_textdomain_just_in_time was called incorrectly. Translation loading for the wpforms-lite domain was triggered too early. This is usually an indicator for some code in the plugin or theme running too early. Translations should be loaded at the init action or later. Please see Debugging in WordPress for more information. (This message was added in version 6.7.0.) in /home/statplace/public_html/site/wp-includes/functions.php on line 6114

Notice: Function _load_textdomain_just_in_time was called incorrectly. Translation loading for the wordpress-seo domain was triggered too early. This is usually an indicator for some code in the plugin or theme running too early. Translations should be loaded at the init action or later. Please see Debugging in WordPress for more information. (This message was added in version 6.7.0.) in /home/statplace/public_html/site/wp-includes/functions.php on line 6114

Notice: A função _load_textdomain_just_in_time foi chamada incorretamente. O carregamento da tradução para o domínio astra foi ativado muito cedo. Isso geralmente é um indicador de que algum código no plugin ou tema está sendo executado muito cedo. As traduções devem ser carregadas na ação init ou mais tarde. Leia como Depurar o WordPress para mais informações. (Esta mensagem foi adicionada na versão 6.7.0.) in /home/statplace/public_html/site/wp-includes/functions.php on line 6114

Warning: Cannot modify header information - headers already sent by (output started at /home/statplace/public_html/site/wp-includes/functions.php:6114) in /home/statplace/public_html/site/wp-includes/rest-api/class-wp-rest-server.php on line 1893

Warning: Cannot modify header information - headers already sent by (output started at /home/statplace/public_html/site/wp-includes/functions.php:6114) in /home/statplace/public_html/site/wp-includes/rest-api/class-wp-rest-server.php on line 1893

Warning: Cannot modify header information - headers already sent by (output started at /home/statplace/public_html/site/wp-includes/functions.php:6114) in /home/statplace/public_html/site/wp-includes/rest-api/class-wp-rest-server.php on line 1893

Warning: Cannot modify header information - headers already sent by (output started at /home/statplace/public_html/site/wp-includes/functions.php:6114) in /home/statplace/public_html/site/wp-includes/rest-api/class-wp-rest-server.php on line 1893

Warning: Cannot modify header information - headers already sent by (output started at /home/statplace/public_html/site/wp-includes/functions.php:6114) in /home/statplace/public_html/site/wp-includes/rest-api/class-wp-rest-server.php on line 1893

Warning: Cannot modify header information - headers already sent by (output started at /home/statplace/public_html/site/wp-includes/functions.php:6114) in /home/statplace/public_html/site/wp-includes/rest-api/class-wp-rest-server.php on line 1893

Warning: Cannot modify header information - headers already sent by (output started at /home/statplace/public_html/site/wp-includes/functions.php:6114) in /home/statplace/public_html/site/wp-includes/rest-api/class-wp-rest-server.php on line 1893

Warning: Cannot modify header information - headers already sent by (output started at /home/statplace/public_html/site/wp-includes/functions.php:6114) in /home/statplace/public_html/site/wp-includes/rest-api/class-wp-rest-server.php on line 1893
{"id":6852,"date":"2020-08-20T13:00:11","date_gmt":"2020-08-20T16:00:11","guid":{"rendered":"https:\/\/site.statplace.com.br\/?p=6852"},"modified":"2024-10-07T15:52:24","modified_gmt":"2024-10-07T15:52:24","slug":"uma-visao-geral-sobre-machine-learning-regressao-clustering","status":"publish","type":"post","link":"https:\/\/site.statplace.com.br\/blog\/uma-visao-geral-sobre-machine-learning-regressao-clustering\/","title":{"rendered":"Uma vis\u00e3o geral sobre machine learning: Regress\u00e3o e Clustering"},"content":{"rendered":"\n

Artigo escrito com a colabora\u00e7\u00e3o de Luis Henrique Velasquez.<\/p>\n\n\n\n

De forma bem simples, o Machine Learning se baseia na constru\u00e7\u00e3o e no uso de algoritmos que \u201caprendem\u201d a partir dos dados.<\/p>\n\n\n\n

Os problemas de Machine Learning s\u00e3o divididos em tr\u00eas sub\u00e1reas principais:<\/strong> classifica\u00e7\u00e3o, regress\u00e3o e clustering. J\u00e1 falamos aqui no blog sobre a Classifica\u00e7\u00e3o<\/a> e no artigo de hoje vamos continuar explicando Regress\u00e3o e Clustering. <\/p>\n\n\n\n

Machine Learning \u00e9 um assunto vasto e que se aproxima muito da estat\u00edstica<\/a> j\u00e1 que os modelos dependem muito das t\u00e9cnicas estat\u00edsticas bem desenvolvidas. <\/p>\n\n\n\n

Regress\u00e3o<\/strong><\/h2>\n\n\n\n
A Regress\u00e3o \u00e9 um dos m\u00e9todos de previs\u00e3o mais utilizados no meio estat\u00edstico. Tem como principal objetivo verificar como certas vari\u00e1veis de interesse influenciam uma vari\u00e1vel resposta Y e criar um modelo matem\u00e1tico capaz de predizer valores de Y com base em novos valores de vari\u00e1veis preditoras X.<\/p>\n\n\n\n
Regress\u00e3o Linear Simples<\/strong><\/h3>\n\n\n\n
\n
<\/iframe><\/div>\n<\/div><\/figure>\n\n\n\nO modelo de regress\u00e3o simples<\/strong> pode ser escrito da seguinte forma:<\/p>\n\n\n\nyi = \u03b20 + \u03b21xi + \u03b5i<\/p>\n\n\n\nSendo que os erros s\u00e3o identicamente distribu\u00eddos, seguindo uma distribui\u00e7\u00e3o Normal com m\u00e9dia 0 e vari\u00e2ncia \u03c3\u00b2. (\u03b5i ~ N(0,\u03c3\u00b2)<\/strong>).<\/p>\n\n\n\nO foco desse semin\u00e1rio n\u00e3o ser\u00e1 entrar em muitos detalhes acerca da teoria estat\u00edstica por tr\u00e1s da regress\u00e3o, mas sim mostrar pontos uteis para um melhor uso no R.<\/p>\n\n\n\nOs principais m\u00e9todos utilizados para avaliar o desempenho<\/strong> dos modelos de regress\u00e3o s\u00e3o:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>RMSE<\/strong>: ou\u00a0Raiz do Erro Quadr\u00e1tico M\u00e9dio<\/strong>, \u00e9 comumente usada para expressar a\u00a0acur\u00e1cia<\/strong>\u00a0dos resultados num\u00e9ricos do modelo. Como j\u00e1 explicado na se\u00e7\u00e3o das medidas de desempenho, quando maior seu valor, pior o desempenho do modelo.<\/li>\n\n\n\n<li>R\u00b2<\/strong>: expressa a porcentagem de\u00a0vari\u00e2ncia explicada pelas vari\u00e1veis independentes<\/strong>\u00a0apresentadas no modelo. Quando maior, melhor o desempenho dele. Por\u00e9m, \u00e9 importante verificar a signific\u00e2ncia e a quantidade dessas vari\u00e1veis antes de verificar o R\u00b2 do modelo, dado que um aumento do n\u00famero de vari\u00e1veis acaba por acarretar um aumento dessa medida.<\/li>\n<\/ul>\n\n\n\nO c\u00f3digo abaixo utiliza o banco de dados \u201ckang.nose\u201d para construirmos uma regress\u00e3o linear simples<\/strong>, criando um modelo que prev\u00ea a largura do nariz de cangurus com base em seu comprimento. Realizaram-se predi\u00e7\u00f5es do modelo atrav\u00e9s da fun\u00e7\u00e3o predict()<\/strong><\/em> e c\u00e1lculos para as medidas de desempenho RMSE<\/strong> e R\u00b2<\/strong>.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>kang.nose %>% summary()<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code> nose_width nose_length \n Min. :189,0 Min. :493,0 \n 1st Qu.:220,0 1st Qu.:616,0 \n Median :237,0 Median :677,0 \n Mean :244,2 Mean :687,4 \n 3rd Qu.:272,0 3rd Qu.:755,0 \n Max. :308,0 Max. :864,0 <\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>### Aplicando um regress\u00e3o linear simples - Witdh explicando por Length\nmod.nose <- lm(nose_width ~ nose_length, data= kang.nose)\nmod.nose %>% summary()<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>\nCall:\nlm(formula = nose_width ~ nose_length, data = kang.nose)\n\nResiduals:\n Min 1Q Median 3Q Max \n-29,656 -7,479 2,132 8,229 27,344 \n\nCoefficients:\n Estimate Std. Error t value Pr(>|t|) \n(Intercept) 46,4508 16,2998 2,85 0,00669 \nnose_length 0,2876 0,0235 12,24 1,34e-15 \n---\nSignif. codes: 0 '' 0,001 '' 0,01 '' 0,05 '.' 0,1 ' ' 1\n\nResidual standard error: 14,44 on 43 degrees of freedom\nMultiple R-squared: 0,7769, Adjusted R-squared: 0,7717 \nF-statistic: 149,7 on 1 and 43 DF, p-value: 1,342e-15<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>mod.nose$coefficients # retorna os coeficientes<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>(Intercept) nose_length \n 46,4508136 0,2876124 <\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>new_length <- data.frame(nose_length = c(500, 600, 700)) # Novos dados\npredict(mod.nose, new_length, interval = \"confidence\") # Previs\u00e3o com base nos novos + IC 95%<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code> fit lwr upr\n1 190,2570 180,3698 200,1442\n2 219,0182 213,0172 225,0193\n3 247,7795 243,3970 252,1619<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>RMSE <- function(error) { sqrt(mean(error^2)) } # Fun\u00e7\u00e3o criada para o c\u00e1lculo do RMSE\nRMSE(mod.nose$residuals) # RMSE<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>[1] 14,11701<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>summary(mod.nose)$r.squared # R2<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>[1] 0,7768914<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">Regress\u00e3o Linear Multivariada<\/strong><\/h3>\n\n\n\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<div class=\"ast-oembed-container\"><iframe loading=\"lazy\" title=\"An\u00e1lise de Regress\u00e3o Linear M\u00faltipla - Defini\u00e7\u00e3o\" width=\"500\" height=\"281\" src=\"https:\/\/www.youtube.com\/embed\/1MbQpCNafWc?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><\/div>\n<\/div><\/figure>\n\n\n\nA regress\u00e3o linear multivariada<\/strong> \u00e9 semelhante \u00e0 simples, por\u00e9m com mais de uma vari\u00e1vel preditora. Para esse tipo de regress\u00e3o, no lugar do R\u00b2 utiliza-se o R\u00b2 ajustado<\/strong>, j\u00e1 que ele consegue penalizar o acr\u00e9scimo de vari\u00e1veis ao modelo, fazendo com que n\u00e3o seja diretamente inflacionada nesse contexto.<\/p>\n\n\n\nNo c\u00f3digo abaixo, trabalharemos com o banco \u201cshop.data\u201d, que apresenta as vendas de uma loja com base em diversas caracter\u00edsticas dos seus clientes. Assim como foi realizado com o caso da regress\u00e3o linear simples, realizaremos previs\u00f5es, calcularemos medidas de desempenho (como o R\u00b2 ajustado) e verificaremos as suposi\u00e7\u00f5es dos erros do modelo de regress\u00e3o multivariada.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>shop.data %>% summary()<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code> sales sq_ft inv ads \n Min. : 0,5 Min. :0,500 Min. :102,0 Min. : 2,50 \n 1st Qu.: 98,5 1st Qu.:1,400 1st Qu.:204,0 1st Qu.: 4,80 \n Median :341,0 Median :3,500 Median :382,0 Median : 8,10 \n Mean :286,6 Mean :3,326 Mean :387,5 Mean : 8,10 \n 3rd Qu.:450,5 3rd Qu.:4,750 3rd Qu.:551,0 3rd Qu.:10,95 \n Max. :570,0 Max. :8,600 Max. :788,0 Max. :17,40 \n size_dist comp \n Min. : 1,600 Min. : 0,000 \n 1st Qu.: 4,500 1st Qu.: 4,000 \n Median :11,300 Median : 8,000 \n Mean : 9,693 Mean : 7,741 \n 3rd Qu.:14,050 3rd Qu.:12,000 \n Max. :16,300 Max. :15,000 <\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>### Aplicando uma regress\u00e3o linear multivariada - Sales como vari\u00e1vel resposta\nmod.shop <- lm(sales ~ . , data = shop.data) # \"~.\" mostra que usaremos todas as demais vari\u00e1veis \nmod.shop %>% summary()<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>\nCall:\nlm(formula = sales ~ ., data = shop.data)\n\nResiduals:\n Min 1Q Median 3Q Max \n-26,338 -9,699 -4,496 4,040 41,139 \n\nCoefficients:\n Estimate Std. Error t value Pr(>|t|) \n(Intercept) -18,85941 30,15023 -0,626 0,538372 \nsq_ft 16,20157 3,54444 4,571 0,000166 \ninv 0,17464 0,05761 3,032 0,006347 \nads 11,52627 2,53210 4,552 0,000174 \nsize_dist 13,58031 1,77046 7,671 1,61e-07 \ncomp -5,31097 1,70543 -3,114 0,005249 \n---\nSignif. codes: 0 '*' 0,001 '**' 0,01 '' 0,05 '.' 0,1 ' ' 1\n\nResidual standard error: 17,65 on 21 degrees of freedom\nMultiple R-squared: 0,9932, Adjusted R-squared: 0,9916 \nF-statistic: 611,6 on 5 and 21 DF, p-value: < 2,2e-16<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>mod.shop$coefficients # retorna os coeficientes<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>(Intercept) sq_ft inv ads size_dist comp \n-18,8594073 16,2015726 0,1746352 11,5262679 13,5803127 -5,3109718 <\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>summary(mod.nose)$r.squared # R2 (aumenta com o acr\u00e9scimo de vari\u00e1veis)<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>[1] 0,7768914<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>summary(mod.nose)$adj.r.squared # R2 Ajustado<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>[1] 0,7717028<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>RMSE(mod.shop$residuals) # RMSE<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>[1] 15,56517<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>#### Verificando suposi\u00e7\u00f5es dos res\u00edduos\n\n# Residuos independentemente distribu\u00eddos em torno de 0\nplot(mod.shop$fitted.values, mod.shop$residuals) \nabline(h=0, col=\"red\", lty= 3) <\/code><\/pre>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1073\" height=\"637\" src=\"https:\/\/i1.wp.com\/operdata.com.br\/wp-content\/uploads\/2020\/08\/residuals.png?fit=1030%2C611&ssl=1\" alt=\"scatterplot da regress\u00e3o linear multivariada\n\" class=\"wp-image-6857\" srcset=\"https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/residuals.png 1073w, https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/residuals-300x178.png 300w, https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/residuals-1024x608.png 1024w, https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/residuals-768x456.png 768w\" sizes=\"auto, (max-width: 1073px) 100vw, 1073px\" \/><\/figure>\n\n\n\n<pre class=\"wp-block-code\"><code># Normalidade dos res\u00edduos\nif(!require(car)){ install.packages(\"car\"); require(car) }\nqqPlot(mod.shop$residuals)<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1039\" height=\"635\" src=\"https:\/\/i1.wp.com\/operdata.com.br\/wp-content\/uploads\/2020\/08\/residuals2.png?fit=1030%2C629&ssl=1\" alt=\"scatterplot da regress\u00e3o linear multivariada\" class=\"wp-image-6858\" srcset=\"https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/residuals2.png 1039w, https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/residuals2-300x183.png 300w, https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/residuals2-1024x626.png 1024w, https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/residuals2-768x469.png 768w\" sizes=\"auto, (max-width: 1039px) 100vw, 1039px\" \/><figcaption class=\"wp-element-caption\">Uma vis\u00e3o geral sobre machine learning –<\/figcaption><\/figure>\n\n\n\n<pre class=\"wp-block-code\"><code>[1] 6 17<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>mod.shop$residuals %>% shapiro.test() # Rejeita a hip\u00f3te nula de normalidade<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>\n Shapiro-Wilk normality test\n\ndata: .\nW = 0,91859, p-value = 0,03651<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">K-Nearest Neighbors (KNN)<\/strong><\/h3>\n\n\n\n\u00c9 importante observar que, assim como no m\u00e9todo de Classifica\u00e7\u00e3o, \u00e9 poss\u00edvel utilizar o k-Nearest Neighbors (KNN) como uma forma de regress\u00e3o n\u00e3o-param\u00e9trica<\/strong>.<\/p>\n\n\n\nNo c\u00f3digo abaixo, usaremos um training<\/em> e um test set<\/em> j\u00e1 constru\u00eddos para o banco de dados \u201cworld\u201d, apresentando o cgdp (similar ao PIB) de diversos pa\u00edses e a sua porcentagem de popula\u00e7\u00e3o urbana. Utilizaremos a fun\u00e7\u00e3o knn.reg()<\/strong><\/em> (pacote \u201cFNN\u201d<\/strong>) para fazer previs\u00f5es de \u201ccgdp\u201d (vari\u00e1vel resposta) no test set<\/em> ap\u00f3s seu treinamento no training<\/em> set<\/em>. Utilizaram-se 20% do total de observa\u00e7\u00f5es alocadas no training set<\/em> para a defini\u00e7\u00e3o do n\u00famero k de vizinhos.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># Separando os preditores e a vari\u00e1vel resposta dos sets\nworld.test.x <- world.test[,-1] %>% as.data.frame()\nworld.train.x <- world.train[,-1] %>% as.data.frame()\n\nworld.train.y <- world.train[,1] %>% as.data.frame()\nworld.test.y <- world.test[,1] %>% as.data.frame()\n\n# KNN = knn(\"Training set\",\"test set\", \"classifica\u00e7\u00e3o real do training\", \"K vizinhos consiederados\")\nif(!require(FNN)){ install.packages(\"FNN\"); require(FNN) }\nknn.world = FNN::knn.reg(train = world.train.x, test = world.test.x, y = world.train.y, k = 28) # k=20% total obs training\nknn.world<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>Prediction:\n [1] 2510,690 2966,211 3153,116 4604,018 6490,574 2510,690 6490,574\n [8] 18704,486 2553,522 32266,901 23111,132 3753,158 6532,045 16700,703\n[15] 26601,322 32266,901 29289,700 16277,721 28704,561 28704,561 11953,832\n[22] 24495,555 6490,574 2553,522 32266,901 18704,486 28704,561 29005,821\n[29] 32266,901 32266,901 17764,754 5941,413 3047,800 3120,619 26673,573\n[36] 24495,555 32266,901 32266,901 15285,950 3135,717 3038,616 23083,414\n[43] 12493,242 11982,323 28704,561 5813,593 28704,561 28617,902 6939,055\n[50] 7045,579 32266,901 5961,592 4123,500 12260,437 18704,486 32266,901\n[57] 2975,100 32266,901 2915,870 2553,522 2868,928<\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">Clustering<\/strong><\/h2>\n\n\n\nO Clustering<\/strong> \u00e9 uma sub\u00e1rea do Machine Learning de aprendizado n\u00e3o-supervisionado<\/strong>, visto que suas t\u00e9cnicas n\u00e3o necessitam de r\u00f3tulos de classes (como no caso da classifica\u00e7\u00e3o e regress\u00e3o) para realizar suas previs\u00f5es. Um Cluster<\/strong> \u00e9 um objeto que cont\u00e9m diversas observa\u00e7\u00f5es que s\u00e3o semelhantes entre si dentro desse cluster espec\u00edfico e diferentes de outras observa\u00e7\u00f5es que foram alocadas em outros clusters. Dessa forma, o Clustering<\/strong> nada mais \u00e9 que agrupar os dados em clusters baseado em suas caracter\u00edsticas\/vari\u00e1veis. Como j\u00e1 comentado anteriormente, n\u00e3o existe propriamente uma clusteriza\u00e7\u00e3o certa ou errada dos dados, sendo que o tipo de clusteriza\u00e7\u00e3o utilizada resulta em diferentes tipos de clusters, e a escolha dessas t\u00e9cnicas devem ser previamente analisadas pelo pesquisador.<\/p>\n\n\n\nO objetivo do uso do Clustering \u00e9 uma melhor visualiza\u00e7\u00e3o dos seus dados, estudando padr\u00f5es de comportamento e detectando outliers. Para realizarmos o Clustering, usamos medidas de similaridade dos dados (como a dist\u00e2ncia Euclidiana para os dados num\u00e9ricos ou vari\u00e1veis dummys para vari\u00e1veis categ\u00f3ricas) e t\u00e9cnicas como k-means<\/strong> e Clustering Hier\u00e1rquico<\/strong>, que ser\u00e3o explicadas mais a frente.<\/p>\n\n\n\nPara que seja poss\u00edvel alcan\u00e7ar o melhor desempenho<\/strong> no processo de clusteriza\u00e7\u00e3o, necessitamos que os clusters apresentem a maior similaridade poss\u00edvel entre si em seu interior e que apresentem a maior diferen\u00e7a poss\u00edvel entre si ao serem comparados uns com os outros. Essa similaridade e dissimilaridade podem ser medidas a partir da \u201csoma de quadrados dentro do Cluster\u201d (WSS)<\/strong> e pela \u201csoma de quadrados entre os clusters\u201d (BSS)<\/strong>, que medem a compacidade<\/strong> e a separa\u00e7\u00e3o<\/strong> dos clusters, respectivamente. Suas f\u00f3rmulas s\u00e3o apresentadas na imagem abaixo:<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1407\" height=\"811\" src=\"https:\/\/i2.wp.com\/operdata.com.br\/wp-content\/uploads\/2020\/08\/cluster-sums-of-squares.png?fit=1030%2C594&ssl=1\" alt=\"c\u00e1lculo do WSS e do BSS\" class=\"wp-image-6859\" srcset=\"https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/cluster-sums-of-squares.png 1407w, https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/cluster-sums-of-squares-300x173.png 300w, https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/cluster-sums-of-squares-1024x590.png 1024w, https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/cluster-sums-of-squares-768x443.png 768w\" sizes=\"auto, (max-width: 1407px) 100vw, 1407px\" \/><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\">K-means<\/strong><\/h3>\n\n\n\nA t\u00e9cnica K-means<\/strong> tem como objetivo separar os dados em k clusters diferentes, com base em k-centroides<\/strong>. Seu algoritmo funciona da seguinte forma:<\/p>\n\n\n\n1.<\/strong> Gera-se aleatoriamente<\/strong> k-centroides. 2.<\/strong> Atribu\u00ed-se os dados aos centroides mais pr\u00f3ximos, criando-se clusters. 3.<\/strong> Move-se os centroides para a localiza\u00e7\u00e3o m\u00e9dia de cada um dos k clusters criados no passo 2, colocando-os no centro real desses clusters criados. 4.<\/strong> Repetem-se os passos 2 e 3 n vezes (a escolha do usu\u00e1rio) ou at\u00e9 que n\u00e3o se ocorra mais mudan\u00e7as significativas nas posi\u00e7\u00f5es dos k centroides.<\/p>\n\n\n\nA melhor escolha do \u201ck\u201d<\/strong> (n\u00famero de clusters) \u00e9 baseado na escolha que apresente a minimiza\u00e7\u00e3o do WSS<\/strong>. Por\u00e9m, quanto maior o k, menor tende a ser o WSS. Para resolver esse problema, escolhemos o k quando a sua diminui\u00e7\u00e3o come\u00e7a a ser muito pequena mesmo com o aumento no n\u00famero de clusters. O crit\u00e9rio usualmente utilizado para essa diminui\u00e7\u00e3o \u00e9 escolher K quando a raz\u00e3o entre o WSS e o TSS<\/strong> (que consiste na soma do WSS com o BSS) for menor que 0,2<\/strong>. O Scree Plot<\/strong> apresenta o gr\u00e1fico dessa raz\u00e3o em rela\u00e7\u00e3o a k, e a principal forma gr\u00e1fica de selecionar-se o k ideal: o k ideal ser\u00e1 quando se forma o \u201ccotovelo\u201d<\/strong> da curva apresentada, pois \u00e9 o momento em que a raz\u00e3o come\u00e7a a decrescer de forma mais suave.<\/p>\n\n\n\nA imagem abaixo mostra um exemplo desse gr\u00e1fico: a partir do ponto k = 3<\/strong>, existe uma diminui\u00e7\u00e3o bem menos expressiva da raz\u00e3o WSS\/TSS, sendo assim a melhor escolha para o n\u00famero de clusters nesse caso.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"1167\" height=\"871\" src=\"https:\/\/i1.wp.com\/operdata.com.br\/wp-content\/uploads\/2020\/08\/cluster-scatter.png?fit=1030%2C769&ssl=1\" alt=\"gr\u00e1fico da raz\u00e3o WSS\/TSS\" class=\"wp-image-6860\" style=\"width:588px;height:439px\" srcset=\"https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/cluster-scatter.png 1167w, https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/cluster-scatter-300x224.png 300w, https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/cluster-scatter-1024x764.png 1024w, https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/cluster-scatter-768x573.png 768w\" sizes=\"auto, (max-width: 1167px) 100vw, 1167px\" \/><\/figure>\n\n\n\nO c\u00f3digo abaixo utiliza o banco de dados seeds, por\u00e9m sem sua classifica\u00e7\u00e3o<\/strong> anteriormente apresentada, e utiliza a fun\u00e7\u00e3o kmeans()<\/strong><\/em> (pacote \u201cstats\u201d<\/strong>) para construirmos 3 clusters diferentes. \u00c9 importante salientar que, quando n\u00e3o indicado na fun\u00e7\u00e3o, o R seleciona aleatoriamente o centroide dos dados, o que pode alterar o c\u00e1lculo dos clusters. Al\u00e9m disso, construiremos um algoritmo para a constru\u00e7\u00e3o de um Scree Plot<\/strong>.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># Semente aleat\u00f3ria\nset.seed(100)\n\n# K-means construido para 3 clusters, repetindo 20 vezes (nstart), com centroide aleatoriamente selecionado (\"centers\" n\u00e3o definido)\nseeds_km <- kmeans(seeds,3,nstart=20)\nseeds_km<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>K-means clustering with 3 clusters of sizes 72, 77, 61\n\nCluster means:\n area perimeter compactness length width asymmetry groove_length\n1 14,64847 14,46042 0,8791667 5,563778 3,277903 2,613892 5,192319\n2 11,96442 13,27481 0,8522000 5,229286 2,872922 4,759740 5,088519\n3 18,72180 16,29738 0,8850869 6,208934 3,722672 3,603590 6,066098\n\nClustering vector:\n [1] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 1 1 2 1 1 1 1 1 1 2 1 1 1 1 1 1 1 1 1 1\n [38] 3 1 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 1 1 1 1 1 2 3 3 3 3\n [75] 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 1 3 3 3 3 3 3 3 3 3 3\n[112] 3 3 3 3 3 3 3 3 3 3 3 1 3 1 3 3 3 3 3 3 3 1 1 1 1 3 1 1 1 2 2 2 2 2 2 2 2\n[149] 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 1 2 2 2 2 2\n[186] 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 1 2 2 2 2 2 2 2 2\n\nWithin cluster sum of squares by cluster:\n[1] 218,6106 195,7453 184,1086\n (between_SS \/ total_SS = 78,1 %)\n\nAvailable components:\n\n[1] \"cluster\" \"centers\" \"totss\" \"withinss\" \"tot.withinss\"\n[6] \"betweenss\" \"size\" \"iter\" \"ifault\" <\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>seeds_km$betweenss # BSS<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>[1] 2137,896<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>seeds_km$tot.withinss # WSS<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>[1] 598,4645<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code># Comparando a clusteriza\u00e7\u00e3o realizada com a classifica\u00e7\u00e3o real das sementes\ntable(seeds_km$cluster,seeds_type$seeds_type)<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code> \n 1 2 3\n 1 60 10 2\n 2 9 0 68\n 3 1 60 0<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code># Plotando o comprimento das sementes em fun\u00e7\u00e3o da sua largura e colorindo de acordo com os clusters criados\nplot(seeds$width,seeds$length,col=seeds_km$cluster)<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1008\" height=\"625\" src=\"https:\/\/statplace.com.br\/wp-content\/uploads\/2020\/08\/scatter-seeds.png\" alt=\"scatterplot do comprimento das sementes\" class=\"wp-image-6862\" srcset=\"https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/scatter-seeds.png 1008w, https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/scatter-seeds-300x186.png 300w, https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/scatter-seeds-768x476.png 768w\" sizes=\"auto, (max-width: 1008px) 100vw, 1008px\" \/><\/figure>\n\n\n\n<pre class=\"wp-block-code\"><code># Criando o Scree Plot para as sementes\nRaz\u00e3o.seed <- rep(0,7) # N\u00famero total de clusters a serem testados no Scree Plot\n\nfor (k in 1:7) {\n seeds_km <- kmeans(seeds,nstart=20,k) # Aplicando K-means\n Raz\u00e3o.seed[k] <- seeds_km$tot.withinss\/seeds_km$totss # Salvando a Raz\u00e3o WSS\/TSS para cada k utilizado\n}\n\nplot(Raz\u00e3o.seed,type=\"b\",xlab=\"k\", main=\"Scree Plot para o banco Seeds\") # K = 3 ou 4\nabline(h=0.2, col=\"red\", lty= 3)<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large\"><img decoding=\"async\" src=\"https:\/\/i0.wp.com\/operdata.com.br\/wp-content\/uploads\/2020\/08\/raz\u00e3o-seeds.png?fit=1030%2C671&ssl=1\" alt=\"scree plot para o banco seeds\" class=\"wp-image-6863\"\/><\/figure>\n\n\n\nUma alternativa para se medir o desempenho da clusteriza\u00e7\u00e3o similar ao uso do BSS e WSS \u00e9 o Dunn Index<\/strong>: essa estat\u00edstica apresenta a raz\u00e3o entre a dist\u00e2ncia m\u00ednima entre os clusters e o di\u00e2metro m\u00e1ximo dos clusters. Assim, um valor alto do Dunn Index mostra que os clusters em quest\u00e3o est\u00e3o bem separados entre si e bem compactos. Por\u00e9m, essa estat\u00edstica tem um custo computacional alto (quando maior o n\u00famero de clusters, maior o custo) e deve ser utilizada no pior caso.<\/p>\n\n\n\nO c\u00f3digo abaixo utiliza o banco de dados run_record, apresentando o recorde de diversos tipos de maratonas. Usaremos a fun\u00e7\u00e3o dunn()<\/strong><\/em> (pacote \u201cclValid\u201d<\/strong>) para o c\u00e1lculo do Dunn Index e a fun\u00e7\u00e3o scale()<\/strong><\/em> para o escalonamento das vari\u00e1veis (iremos padronizar os dados, pois a escala entre os recordes se diferenciava substancialmente).<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>summary(run_record) # as escalas dos records se diferenciam muito!<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code> Country X100m X200m X400m X800m \n Argentina: 1 Min. : 9,78 Min. :19,32 Min. :43,18 Min. :101,4 \n Australia: 1 1st Qu.:10,10 1st Qu.:20,17 1st Qu.:44,91 1st Qu.:103,8 \n Austria : 1 Median :10,20 Median :20,43 Median :45,58 Median :105,6 \n Belgium : 1 Mean :10,22 Mean :20,54 Mean :45,83 Mean :106,1 \n Bermuda : 1 3rd Qu.:10,32 3rd Qu.:20,84 3rd Qu.:46,32 3rd Qu.:108,0 \n Brazil : 1 Max. :10,97 Max. :22,46 Max. :51,40 Max. :116,4 \n (Other) :48 \n X1500m X5000m X10000m marathon \n Min. :206,4 Min. : 759,6 Min. :1588 Min. : 7473 \n 1st Qu.:213,0 1st Qu.: 788,9 1st Qu.:1653 1st Qu.: 7701 \n Median :216,6 Median : 805,2 Median :1675 Median : 7819 \n Mean :219,2 Mean : 817,1 Mean :1712 Mean : 8009 \n 3rd Qu.:224,2 3rd Qu.: 834,5 3rd Qu.:1739 3rd Qu.: 8050 \n Max. :254,4 Max. :1002,0 Max. :2123 Max. :10276 \n <\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code># Semente aleatoria\nset.seed(16)\n\n# K-means para valores num\u00e9ricos do run_record: 5 clusters, repetir 20 vezes\nrun_km <- kmeans(run_record[,2:9],nstart=20,5)\n\n# Calculando o Dunn's index\nif(!require(clValid)){ install.packages(\"clValid\"); require(clValid) }\ndunn_km <- dunn(clusters=run_km$cluster,Data=run_record[,2:9])\ndunn_km<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>[1] 0,05651773<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code># Escalonizando os dados\nrun_record_sc <- as.data.frame(scale(run_record[2:9]))\n\n# K-means: Run record escolonado\nrun_km_sc <- kmeans(run_record_sc,nstart = 20,5)\n\n# Comparando os clusters do run record escalonado e n\u00e3o-escalonado\ntable(run_km$cluster,run_km_sc$cluster)<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code> \n 1 2 3 4 5\n 1 7 0 0 1 10\n 2 1 0 0 13 10\n 3 0 0 6 0 0\n 4 0 2 0 0 0\n 5 2 0 2 0 0<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code># Calculando o Dunn index para o run record escalonizado\ndunn_km_sc<-dunn(cluster=run_km_sc$cluster,Data=run_record_sc)\ndunn_km_sc # O dunn escalonado \u00e9 bem superior ao n\u00e3o escalonado<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>[1] 0,1453556<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">Clustering Hier\u00e1rquico<\/strong><\/h3>\n\n\n\nAl\u00e9m da t\u00e9cnica do K-means, outra t\u00e9cnica importante no Clustering \u00e9 o Clustering Hier\u00e1rquico<\/strong>. Diferentemente da t\u00e9cnica K-means, o Clustering Hier\u00e1rquico apresenta a ordem de jun\u00e7\u00e3o<\/strong> dos objetos de cada cluster, assim como quais clusters se fundiram para a forma\u00e7\u00e3o de um novo cluster, seguindo um comportamento hier\u00e1rquico dos clusters. A hierarquia de forma\u00e7\u00e3o dos clusters depender\u00e1 dos tipos de dist\u00e2ncias utilizadas para medir a similaridade entre os objetos. \u00c9 importante observar que essa t\u00e9cnica consiste na forma\u00e7\u00e3o de cluster atrav\u00e9s de passos, e cada cluster \u00e9 formado separadamente<\/strong> seguindo-se uma hierarquia.<\/p>\n\n\n\nO algoritmo utilizado no Clustering Hier\u00e1rquico funciona da seguinte forma:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Antes de tudo, calcula-se a dist\u00e2ncia entre os objetos\/observa\u00e7\u00f5es. 1.<\/strong>\u00a0Coloca-se cada objeto em seu pr\u00f3prio cluster. 2.<\/strong>\u00a0Mescla-se o par de objetos mais pr\u00f3ximos entre si. Demais objetos continuar\u00e3o iguais. 3.<\/strong>\u00a0Recalcula-se a dist\u00e2ncia entre o novo cluster formado e os demais objetos restantes. 4.<\/strong>\u00a0Repetem-se os passos 2 e 3 at\u00e9 um crit\u00e9rio de parada (geralmente, at\u00e9 a forma\u00e7\u00e3o de um \u00fanico cluster final).<\/li>\n<\/ul>\n\n\n\nExistem tr\u00eas m\u00e9todos utilizados para determinar a liga\u00e7\u00e3o entre os objetos e forma\u00e7\u00e3o dos clusters, sendo que cada um deles afeta diretamente na forma\u00e7\u00e3o dos clusters:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Simple-Linkage:<\/strong>\u00a0a dist\u00e2ncia entre dois clusters \u00e9 definida como a dist\u00e2ncia\u00a0m\u00ednima<\/strong>\u00a0encontrada entre os objetos de um dos clusters para os demais objetos do outro cluster.<\/li>\n\n\n\n<li>Complete-Linkage:<\/strong>\u00a0a dist\u00e2ncia entre dois clusters \u00e9 definida como a dist\u00e2ncia\u00a0m\u00e1xima<\/strong>\u00a0encontrada entre os objetos de um dos clusters para os demais objetos do outro cluster.<\/li>\n\n\n\n<li>Average-Linkage:<\/strong>\u00a0a dist\u00e2ncia entre dois clusters \u00e9 definida como a dist\u00e2ncia\u00a0m\u00e9dia<\/strong>\u00a0encontrada entre os objetos dos dois clusters.<\/li>\n<\/ol>\n\n\n\nAp\u00f3s a escolha do m\u00e9todo de linkagem, constru\u00edmos um gr\u00e1fico chamado Dendrograma<\/strong>. Esse gr\u00e1fico consegue nos mostrar a ordem das mesclas<\/strong> entre os objetos (barras horizontais), assim como a dist\u00e2ncia entre os clusters<\/strong> antes deles serem mesclados (altura do dendrograma). Pode-se \u201ccortar\u201d o dendrograma de acordo com o n\u00famero de clusters desejado pelo pesquisador. A imagem abaixo nos d\u00e1 a ideia de como esse gr\u00e1fico \u00e9 apresentado:<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1371\" height=\"723\" src=\"https:\/\/i2.wp.com\/operdata.com.br\/wp-content\/uploads\/2020\/08\/folhas-objetos.png?fit=1030%2C543&ssl=1\" alt=\"exemplo de dendograma de clustering hier\u00e1rquico \" class=\"wp-image-6864\" srcset=\"https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/folhas-objetos.png 1371w, https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/folhas-objetos-300x158.png 300w, https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/folhas-objetos-1024x540.png 1024w, https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/folhas-objetos-768x405.png 768w\" sizes=\"auto, (max-width: 1371px) 100vw, 1371px\" \/><\/figure>\n\n\n\nO c\u00f3digo abaixo apresente a aplica\u00e7\u00e3o da t\u00e9cnica de Clustering Hier\u00e1rquico no banco de dados \u201cseeds\u201d, que apresenta diversas caracter\u00edsticas de 3 tipos de sementes, assim como sua classifica\u00e7\u00e3o. Iremos utilizar esse banco de dados retirando a coluna referente a esp\u00e9cie das sementes<\/strong> para realizarmos previs\u00f5es para suas classifica\u00e7\u00f5es e depois as compararemos com suas classifica\u00e7\u00f5es originais. Utilizaremos a fun\u00e7\u00e3o hclust()<\/strong><\/em> (pacote \u201cstats\u201d<\/strong>) para realizar o clustering hier\u00e1rquico e a fun\u00e7\u00e3o cutree()<\/strong><\/em> (pacote \u201cdendextend\u201d<\/strong>) para o corte do dendrograma no n\u00famero de clusters desejado (k = 3, referente ao n\u00famero de esp\u00e9cies originalmente classificadas).<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>summary(seeds) # Analisando o banco de dados - verificando as escalas das variaveis<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code> area perimeter compactness length \n Min. :10,59 Min. :12,41 Min. :0,8081 Min. :4,899 \n 1st Qu.:12,27 1st Qu.:13,45 1st Qu.:0,8569 1st Qu.:5,262 \n Median :14,36 Median :14,32 Median :0,8734 Median :5,524 \n Mean :14,85 Mean :14,56 Mean :0,8710 Mean :5,629 \n 3rd Qu.:17,30 3rd Qu.:15,71 3rd Qu.:0,8878 3rd Qu.:5,980 \n Max. :21,18 Max. :17,25 Max. :0,9183 Max. :6,675 \n width asymmetry groove_length \n Min. :2,630 Min. :0,000077 Min. :4,519 \n 1st Qu.:2,944 1st Qu.:2,561500 1st Qu.:5,045 \n Median :3,237 Median :3,599000 Median :5,223 \n Mean :3,259 Mean :3,688187 Mean :5,408 \n 3rd Qu.:3,562 3rd Qu.:4,768750 3rd Qu.:5,877 \n Max. :4,033 Max. :8,456000 Max. :6,550 <\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>#### Como as vari\u00e1veis estavam em escalas diferentes, escalonamos ela a partir da normaliza\u00e7\u00e3o\nseeds_sc <- seeds %>% scale %>% as.data.frame() \nsummary(seeds_sc) # Todas escolonadas com m\u00e9dia 0 e dp 1<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code> area perimeter compactness length \n Min. :-1,4632 Min. :-1,6458 Min. :-2,6619 Min. :-1,6466 \n 1st Qu.:-0,8858 1st Qu.:-0,8494 1st Qu.:-0,5967 1st Qu.:-0,8267 \n Median :-0,1693 Median :-0,1832 Median : 0,1037 Median :-0,2371 \n Mean : 0,0000 Mean : 0,0000 Mean : 0,0000 Mean : 0,0000 \n 3rd Qu.: 0,8446 3rd Qu.: 0,8850 3rd Qu.: 0,7100 3rd Qu.: 0,7927 \n Max. : 2,1763 Max. : 2,0603 Max. : 2,0018 Max. : 2,3619 \n width asymmetry groove_length \n Min. :-1,6642 Min. :-2,41116 Min. :-1,8090 \n 1st Qu.:-0,8329 1st Qu.:-0,73659 1st Qu.:-0,7387 \n Median :-0,0572 Median :-0,05831 Median :-0,3766 \n Mean : 0,0000 Mean : 0,00000 Mean : 0,0000 \n 3rd Qu.: 0,8026 3rd Qu.: 0,70644 3rd Qu.: 0,9541 \n Max. : 2,0502 Max. : 3,11704 Max. : 2,3234 <\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>### Calculando a dist\u00e2ncia entre os objetos (dist\u00e2ncia Euclidiana)\ndist_mat <- dist(seeds_sc, method = 'euclidean')\n\n#### Criando o dendrograma e testando cada um dos tr\u00eas m\u00e9todos de liga\u00e7\u00e3o\nif(!require(stats)){ install.packages(\"stats\"); require(stats) }\n\nhclust_single <- hclust(dist_mat, method = 'single') # Usa a Distancia m\u00ednima entre objetos\nplot(hclust_single)<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1009\" height=\"695\" src=\"https:\/\/statplace.com.br\/wp-content\/uploads\/2020\/08\/cluster-dendogram.png\" alt=\"dendograma de clustering\" class=\"wp-image-6865\" srcset=\"https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/cluster-dendogram.png 1009w, https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/cluster-dendogram-300x207.png 300w, https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/cluster-dendogram-768x529.png 768w\" sizes=\"auto, (max-width: 1009px) 100vw, 1009px\" \/><\/figure>\n\n\n\n<pre class=\"wp-block-code\"><code>hclust_complete <- hclust(dist_mat, method = 'complete') # Usa a Distancia m\u00e1xima entre objetos\nplot(hclust_complete)<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1065\" height=\"705\" src=\"https:\/\/i2.wp.com\/operdata.com.br\/wp-content\/uploads\/2020\/08\/cluster-dendogram2.png?fit=1030%2C682&ssl=1\" alt=\"dendograma de clustering\" class=\"wp-image-6866\" srcset=\"https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/cluster-dendogram2.png 1065w, https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/cluster-dendogram2-300x199.png 300w, https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/cluster-dendogram2-1024x678.png 1024w, https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/cluster-dendogram2-768x508.png 768w\" sizes=\"auto, (max-width: 1065px) 100vw, 1065px\" \/><\/figure>\n\n\n\n<pre class=\"wp-block-code\"><code>hclust_avg <- hclust(dist_mat, method = 'average') # Usa a Distancia m\u00e9dia entre objetos\nplot(hclust_avg)<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1045\" height=\"695\" src=\"https:\/\/i0.wp.com\/operdata.com.br\/wp-content\/uploads\/2020\/08\/cluster-dendogram3.png?fit=1030%2C685&ssl=1\" alt=\"dendograma de clustering\" class=\"wp-image-6867\" srcset=\"https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/cluster-dendogram3.png 1045w, https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/cluster-dendogram3-300x200.png 300w, https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/cluster-dendogram3-1024x681.png 1024w, https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/cluster-dendogram3-768x511.png 768w\" sizes=\"auto, (max-width: 1045px) 100vw, 1045px\" \/><\/figure>\n\n\n\n<pre class=\"wp-block-code\"><code>### Dado que ja sabemos que os dados originais continham 3 tipos de sementes, vamos cortar o gr\u00e1fico com k=3\nif(!require(dendextend)){ install.packages(\"dendextend\"); require(dendextend) }\n\ncut_avg <- cutree(hclust_avg, k = 3) # Cortando o dendrograma que utilizou o m\u00e9dio \"average\"\n\n### Colorindo os tr\u00eas clusters formados no dendograma via m\u00e9todo \"average\"\navg_col_dend <- hclust_avg %>% as.dendrogram() %>% color_branches(h = 3) # Fun\u00e7\u00e3o que colore os clusters\nplot(avg_col_dend)<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"959\" height=\"531\" src=\"https:\/\/statplace.com.br\/wp-content\/uploads\/2020\/08\/cluster-dendogram4.png\" alt=\"dendograma de clustering\" class=\"wp-image-6868\" srcset=\"https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/cluster-dendogram4.png 959w, https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/cluster-dendogram4-300x166.png 300w, https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/cluster-dendogram4-768x425.png 768w\" sizes=\"auto, (max-width: 959px) 100vw, 959px\" \/><\/figure>\n\n\n\n<pre class=\"wp-block-code\"><code>### Comparando com os clusters com seus rotulos originais\nseeds_sc <- seeds_sc %>% mutate(cluster = cut_avg)\nconf.seeds <- table(seeds_sc$cluster,seeds_type$seeds_type) # Cluster originais nas colunas\nconf.seeds<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code> \n 1 2 3\n 1 52 2 8\n 2 10 68 0\n 3 8 0 62<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>accuracy <- function(x){sum(diag(x)\/(sum(rowSums(x)))) 100} # Fun\u00e7\u00e3o para calcular a acur\u00e1cia da previs\u00e3o\naccuracy(conf.seeds) # 86,67 % de acur\u00e1cia <\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>[1] 86,66667<\/code><\/pre>\n\n\n\n<\/p>\n","protected":false},"excerpt":{"rendered":"Machine Learning \u00e9 uma intelig\u00eancia artificial que permite que computadores tomem decis\u00f5es com a ajuda de algoritmos. Neste artigo vamos falar de regress\u00e3o e clustering.<\/p>\n","protected":false},"author":8,"featured_media":6855,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"site-sidebar-layout":"","site-content-layout":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","spay_email":"","footnotes":""},"categories":[442,444],"tags":[257,255,124,192,89],"class_list":["post-6852","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-fundamentos-de-estatistica","category-machine-learning","tag-arvore-de-decisao","tag-clustering","tag-machine-learning","tag-modelos-estatisticos","tag-regressao-linear"],"yoast_head":"\n<title>Uma vis\u00e3o geral sobre machine learning: Regress\u00e3o e Clustering - Statplace<\/title>\n<meta name=\"description\" content=\"Explore uma vis\u00e3o geral sobre Machine Learning, com foco em t\u00e9cnicas de Regress\u00e3o e Clustering, para resolver problemas complexos de dados.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/statplace.com.br\/blog\/uma-visao-geral-sobre-machine-learning-regressao-clustering\/\" \/>\n<meta property=\"og:locale\" content=\"pt_BR\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Uma vis\u00e3o geral sobre machine learning: Regress\u00e3o e Clustering - Statplace\" \/>\n<meta property=\"og:description\" content=\"Explore uma vis\u00e3o geral sobre Machine Learning, com foco em t\u00e9cnicas de Regress\u00e3o e Clustering, para resolver problemas complexos de dados.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/statplace.com.br\/blog\/uma-visao-geral-sobre-machine-learning-regressao-clustering\/\" \/>\n<meta property=\"og:site_name\" content=\"Statplace\" \/>\n<meta property=\"article:published_time\" content=\"2020-08-20T16:00:11+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-10-07T15:52:24+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/statplace.com.br\/wp-content\/uploads\/2020\/08\/artigoml2-20-scaled-1.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"2560\" \/>\n\t<meta property=\"og:image:height\" content=\"920\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Escrito por\" \/>\n\t<meta name=\"twitter:data1\" content=\"Adilane Ribeiro da Silva\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. tempo de leitura\" \/>\n\t<meta name=\"twitter:data2\" content=\"12 minutos\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebSite\",\"@id\":\"https:\/\/statplace.com.br\/#website\",\"url\":\"https:\/\/statplace.com.br\/\",\"name\":\"Statplace\",\"description\":\"A Estat\u00edstica ao alcance de todos.\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/statplace.com.br\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"pt-BR\"},{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/statplace.com.br\/blog\/uma-visao-geral-sobre-machine-learning-regressao-clustering\/#primaryimage\",\"inLanguage\":\"pt-BR\",\"url\":\"https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/artigoml2-20-scaled-1.jpg\",\"contentUrl\":\"https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/artigoml2-20-scaled-1.jpg\",\"width\":2560,\"height\":920},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/statplace.com.br\/blog\/uma-visao-geral-sobre-machine-learning-regressao-clustering\/#webpage\",\"url\":\"https:\/\/statplace.com.br\/blog\/uma-visao-geral-sobre-machine-learning-regressao-clustering\/\",\"name\":\"Uma vis\u00e3o geral sobre machine learning: Regress\u00e3o e Clustering - Statplace\",\"isPartOf\":{\"@id\":\"https:\/\/statplace.com.br\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/statplace.com.br\/blog\/uma-visao-geral-sobre-machine-learning-regressao-clustering\/#primaryimage\"},\"datePublished\":\"2020-08-20T16:00:11+00:00\",\"dateModified\":\"2024-10-07T15:52:24+00:00\",\"author\":{\"@id\":\"https:\/\/statplace.com.br\/#\/schema\/person\/4e381a96f090496264e98c9be9afddc9\"},\"description\":\"Explore uma vis\u00e3o geral sobre Machine Learning, com foco em t\u00e9cnicas de Regress\u00e3o e Clustering, para resolver problemas complexos de dados.\",\"breadcrumb\":{\"@id\":\"https:\/\/statplace.com.br\/blog\/uma-visao-geral-sobre-machine-learning-regressao-clustering\/#breadcrumb\"},\"inLanguage\":\"pt-BR\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/statplace.com.br\/blog\/uma-visao-geral-sobre-machine-learning-regressao-clustering\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/statplace.com.br\/blog\/uma-visao-geral-sobre-machine-learning-regressao-clustering\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"In\u00edcio\",\"item\":\"https:\/\/statplace.com.br\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Uma vis\u00e3o geral sobre machine learning: Regress\u00e3o e Clustering\"}]},{\"@type\":\"Person\",\"@id\":\"https:\/\/statplace.com.br\/#\/schema\/person\/4e381a96f090496264e98c9be9afddc9\",\"name\":\"Adilane Ribeiro da Silva\",\"image\":{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/statplace.com.br\/#personlogo\",\"inLanguage\":\"pt-BR\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/cba8d3cd5f938f2bda6868e657a95cb7?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/cba8d3cd5f938f2bda6868e657a95cb7?s=96&d=mm&r=g\",\"caption\":\"Adilane Ribeiro da Silva\"},\"url\":\"https:\/\/site.statplace.com.br\/blog\/author\/adilane\/\"}]}<\/script>\n","yoast_head_json":{"title":"Uma vis\u00e3o geral sobre machine learning: Regress\u00e3o e Clustering - Statplace","description":"Explore uma vis\u00e3o geral sobre Machine Learning, com foco em t\u00e9cnicas de Regress\u00e3o e Clustering, para resolver problemas complexos de dados.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/statplace.com.br\/blog\/uma-visao-geral-sobre-machine-learning-regressao-clustering\/","og_locale":"pt_BR","og_type":"article","og_title":"Uma vis\u00e3o geral sobre machine learning: Regress\u00e3o e Clustering - Statplace","og_description":"Explore uma vis\u00e3o geral sobre Machine Learning, com foco em t\u00e9cnicas de Regress\u00e3o e Clustering, para resolver problemas complexos de dados.","og_url":"https:\/\/statplace.com.br\/blog\/uma-visao-geral-sobre-machine-learning-regressao-clustering\/","og_site_name":"Statplace","article_published_time":"2020-08-20T16:00:11+00:00","article_modified_time":"2024-10-07T15:52:24+00:00","og_image":[{"width":2560,"height":920,"url":"https:\/\/statplace.com.br\/wp-content\/uploads\/2020\/08\/artigoml2-20-scaled-1.jpg","type":"image\/jpeg"}],"twitter_card":"summary_large_image","twitter_misc":{"Escrito por":"Adilane Ribeiro da Silva","Est. tempo de leitura":"12 minutos"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebSite","@id":"https:\/\/statplace.com.br\/#website","url":"https:\/\/statplace.com.br\/","name":"Statplace","description":"A Estat\u00edstica ao alcance de todos.","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/statplace.com.br\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"pt-BR"},{"@type":"ImageObject","@id":"https:\/\/statplace.com.br\/blog\/uma-visao-geral-sobre-machine-learning-regressao-clustering\/#primaryimage","inLanguage":"pt-BR","url":"https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/artigoml2-20-scaled-1.jpg","contentUrl":"https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/artigoml2-20-scaled-1.jpg","width":2560,"height":920},{"@type":"WebPage","@id":"https:\/\/statplace.com.br\/blog\/uma-visao-geral-sobre-machine-learning-regressao-clustering\/#webpage","url":"https:\/\/statplace.com.br\/blog\/uma-visao-geral-sobre-machine-learning-regressao-clustering\/","name":"Uma vis\u00e3o geral sobre machine learning: Regress\u00e3o e Clustering - Statplace","isPartOf":{"@id":"https:\/\/statplace.com.br\/#website"},"primaryImageOfPage":{"@id":"https:\/\/statplace.com.br\/blog\/uma-visao-geral-sobre-machine-learning-regressao-clustering\/#primaryimage"},"datePublished":"2020-08-20T16:00:11+00:00","dateModified":"2024-10-07T15:52:24+00:00","author":{"@id":"https:\/\/statplace.com.br\/#\/schema\/person\/4e381a96f090496264e98c9be9afddc9"},"description":"Explore uma vis\u00e3o geral sobre Machine Learning, com foco em t\u00e9cnicas de Regress\u00e3o e Clustering, para resolver problemas complexos de dados.","breadcrumb":{"@id":"https:\/\/statplace.com.br\/blog\/uma-visao-geral-sobre-machine-learning-regressao-clustering\/#breadcrumb"},"inLanguage":"pt-BR","potentialAction":[{"@type":"ReadAction","target":["https:\/\/statplace.com.br\/blog\/uma-visao-geral-sobre-machine-learning-regressao-clustering\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/statplace.com.br\/blog\/uma-visao-geral-sobre-machine-learning-regressao-clustering\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"In\u00edcio","item":"https:\/\/statplace.com.br\/"},{"@type":"ListItem","position":2,"name":"Uma vis\u00e3o geral sobre machine learning: Regress\u00e3o e Clustering"}]},{"@type":"Person","@id":"https:\/\/statplace.com.br\/#\/schema\/person\/4e381a96f090496264e98c9be9afddc9","name":"Adilane Ribeiro da Silva","image":{"@type":"ImageObject","@id":"https:\/\/statplace.com.br\/#personlogo","inLanguage":"pt-BR","url":"https:\/\/secure.gravatar.com\/avatar\/cba8d3cd5f938f2bda6868e657a95cb7?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/cba8d3cd5f938f2bda6868e657a95cb7?s=96&d=mm&r=g","caption":"Adilane Ribeiro da Silva"},"url":"https:\/\/site.statplace.com.br\/blog\/author\/adilane\/"}]}},"jetpack_featured_media_url":"https:\/\/site.statplace.com.br\/wp-content\/uploads\/2020\/08\/artigoml2-20-scaled-1.jpg","_links":{"self":[{"href":"https:\/\/site.statplace.com.br\/wp-json\/wp\/v2\/posts\/6852","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/site.statplace.com.br\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/site.statplace.com.br\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/site.statplace.com.br\/wp-json\/wp\/v2\/users\/8"}],"replies":[{"embeddable":true,"href":"https:\/\/site.statplace.com.br\/wp-json\/wp\/v2\/comments?post=6852"}],"version-history":[{"count":2,"href":"https:\/\/site.statplace.com.br\/wp-json\/wp\/v2\/posts\/6852\/revisions"}],"predecessor-version":[{"id":27870,"href":"https:\/\/site.statplace.com.br\/wp-json\/wp\/v2\/posts\/6852\/revisions\/27870"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/site.statplace.com.br\/wp-json\/wp\/v2\/media\/6855"}],"wp:attachment":[{"href":"https:\/\/site.statplace.com.br\/wp-json\/wp\/v2\/media?parent=6852"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/site.statplace.com.br\/wp-json\/wp\/v2\/categories?post=6852"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/site.statplace.com.br\/wp-json\/wp\/v2\/tags?post=6852"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}