Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Assignment 4 Submission #200

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
.Rproj.user
.Rhistory
.RData
.Ruserdata
107 changes: 94 additions & 13 deletions Assignment 4.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,15 @@ https://www.cs.uic.edu/~wilkinson/Applets/cluster.html


```{r}
library()
library(tidyverse)
library(tidyr)
library(dplyr)
```

Now, upload the file "Class_Motivation.csv" from the Assignment 4 Repository as a data frame called "K1""
```{r}

K1 <- read.csv(...)
K1 <- read.csv("HUDK405020-cluster.csv", header = T)

```

Expand All @@ -26,7 +28,7 @@ The algorithm will treat each row as a value belonging to a person, so we need t

```{r}

K2 <-
K2 <- select(K1, 2:6)

```

Expand All @@ -39,15 +41,17 @@ We will remove people with missing values for this assignment, but keep in mind

```{r}

K3 <- na.omit(K2) #This command create a data frame with only those people with no missing values. It "omits" all rows with missing values, also known as a "listwise deletion". EG - It runs down the list deleting rows as it goes.
K3 <- na.omit(K2)

#This command create a data frame with only those people with no missing values. It "omits" all rows with missing values, also known as a "listwise deletion". EG - It runs down the list deleting rows as it goes.

```

Another pre-processing step used in K-means is to standardize the values so that they have the same range. We do this because we want to treat each week as equally important - if we do not standardise then the week with the largest range will have the greatest impact on which clusters are formed. We standardise the values by using the "scale()" command.

```{r}

K3 <-
K3 <- as.data.frame(scale(K3))

```

Expand All @@ -66,20 +70,23 @@ Also, we need to choose the number of clusters we think are in the data. We will

```{r}

fit <-
fit1 <- kmeans(K3, 2)

#We have created an object called "fit" that contains all the details of our clustering including which observations belong to each cluster.

#We can access the list of clusters by typing "fit$cluster", the top row corresponds to the original order the rows were in. Notice we have deleted some rows.


fit1$cluster

#We can also attach these clusters to the original dataframe by using the "data.frame" command to create a new data frame called K4.

K4
K4 <- data.frame(K3, fit1$cluster)

#Have a look at the K4 dataframe. Lets change the names of the variables to make it more convenient with the names() command.

K4 <- data.frame(K3, fit1$cluster)
rownames(K4) <- seq(1, nrow(K4), 1)
colnames(K4) <- c(1:5, "cluster")

```

Expand All @@ -89,13 +96,14 @@ First lets use tidyr to convert from wide to long format.
```{r}

K5 <- gather(K4, "week", "motivation", 1:5)

```

Now lets use dplyr to average our motivation values by week and by cluster.

```{r}

K6 <- K5 %>% group_by(week, cluster) %>% summarise(K6, avg = mean(motivation))
K6 <- K5 %>% group_by(week, cluster) %>% summarise(avg = mean(motivation))

```

Expand All @@ -113,9 +121,8 @@ Likewise, since "cluster" is not numeric but rather a categorical label we want

```{r}

K6$week <-

K6$cluster <-
K6$week <- as.numeric(K6$week)
K6$cluster <- as.factor(K6$cluster)

```

Expand All @@ -134,26 +141,100 @@ ggplot(K6, aes(week, avg, colour = cluster)) + geom_line() + xlab("Week") + ylab

What patterns do you see in the plot?

```{r}

# ANS
# There are two different polylines, each stands for cluster1 and cluster2.
# The two clusters behave in an opposite manner in the 5 weeks. When average motivation of cluster 1 increases, average motivation of cluster2 decreses, vice versa.
# The polyline is alternating for both clusters, if they increase for one week then it will decrease for the next.

```

It would be useful to determine how many people are in each cluster. We can do this easily with dplyr.

```{r}
K7 <- count(K4, cluster)
K7 <- dplyr::count(K4, cluster)
```

Look at the number of people in each cluster, now repeat this process for 3 rather than 2 clusters. Which cluster grouping do you think is more informative? Write your answer below:

```{r}

fit1 <- kmeans(K3, 3)
K4_2 <- data.frame(K3, fit1$cluster)
rownames(K4_2) <- seq(1,nrow(K4_2),1)
colnames(K4_2) <- c(1:5, "cluster")

K5_2 <- gather(K4_2, "week", "motivation", 1:5)

K6_2 <- K5_2 %>% group_by(week, cluster) %>% summarise(avg = mean(motivation))

K6_2$week <- as.numeric(K6_2$week)
K6_2$cluster <- as.factor(K6_2$cluster)

ggplot(K6_2, aes(week, avg, colour = cluster)) + geom_line() + xlab("Week") + ylab("Average Motivation")

# I think the three cluster grouping is more informative since it is more suitable for the data. Clusters are able to capture more characteristics of the given data and give more meaningful and different patterns of each cluster.

```

##Part II

Using the data collected in the HUDK4050 entrance survey (HUDK4050-cluster.csv) use K-means to cluster the students first according location (lat/long) and then according to their answers to the questions, each student should belong to two clusters.

```{r}

library(tidyverse)
M1 <- read.csv("HUDK405020-cluster.csv", header = T)
M2 <- select(M1,4:9)
M2[M2==""] <- NA
M2 <- na.omit(M2)

M2 <- as.data.frame(scale(M2))
fit2a <- kmeans(M2, 1)
fit2b <- kmeans(M2, 2)
fit2c <- kmeans(M2, 3)
fit2d <- kmeans(M2, 4)
fit2e <- kmeans(M2, 5)
fit2f <- kmeans(M2, 6)
fit2g <- kmeans(M2, 7)
mss<- c(fit2a$tot.withinss,fit2b$tot.withinss,fit2c$tot.withinss,fit2d$tot.withinss,fit2e$tot.withinss,fit2f$tot.withinss,fit2g$tot.withinss, fit2a$betweenss,fit2b$betweenss,fit2c$betweenss,fit2d$betweenss,fit2e$betweenss,fit2f$betweenss,fit2g$betweenss)
clusters <- c(seq(1,7,1),seq(1,7,1))
col <- c(rep("blue",7), rep("red",7))
plot(clusters, mss, col = col)

L1 <- select(M1, 2:3)
plot(L1$long, L1$lat)

fit3a <- kmeans(L1, 2)
fit3b <- kmeans(L1, 2)
fit3c <- kmeans(L1, 2)
fit3a$tot.withinss
fit3b$tot.withinss
fit3c$tot.withinss

ML <- data.frame(M1$compare.features, M1$math.accuracy,M1$planner.use,M1$enjoy.discuss,M1$enjoy.group,M1$meet.deadline, fit2c$cluster, M1$lat,M1$long, fit3a$cluster)
pairs(ML)

```

##Part III

Create a visualization that shows the overlap between the two clusters each student belongs to in Part II. IE - Are there geographical patterns that correspond to the answers?

```{r}

DF <- data.frame(table(ML$fit2c.cluster,ML$fit3a.cluster))

ML2 <- ML %>% group_by(fit2c.cluster,fit3a.cluster) %>% summarize(count = n())
ML2$fit3a.cluster <- ifelse(ML2$fit3a.cluster == 1, "A","B")
ggplot(ML2, aes(x = fit2c.cluster, y = fit3a.cluster, size = count)) + geom_point()
geom_bar(stat = "identity", position = "fill", colour = "black")

library(vcd)
P1 <- structable(fit2c$cluster ~ fit3a$cluster)
mosaic(P1, shade=TRUE, legend=TRUE)

```


Expand Down
577 changes: 577 additions & 0 deletions Assignment-4.html

Large diffs are not rendered by default.

13 changes: 13 additions & 0 deletions Assignment4.Rproj
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
Version: 1.0

RestoreWorkspace: Default
SaveWorkspace: Default
AlwaysSaveHistory: Default

EnableCodeIndexing: Yes
UseSpacesForTab: Yes
NumSpacesForTab: 2
Encoding: UTF-8

RnwWeave: Sweave
LaTeX: pdfLaTeX