-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy path3-wrangling.html
1699 lines (1636 loc) · 171 KB
/
3-wrangling.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE html>
<html lang="" xml:lang="">
<head>
<meta charset="utf-8" />
<meta http-equiv="X-UA-Compatible" content="IE=edge" />
<title>Chapter 3 Data Wrangling | Statistical Inference via Data Science</title>
<meta name="description" content="An open-source and fully-reproducible electronic textbook for teaching statistical inference using tidyverse data science tools." />
<meta name="generator" content="bookdown 0.22 and GitBook 2.6.7" />
<meta property="og:title" content="Chapter 3 Data Wrangling | Statistical Inference via Data Science" />
<meta property="og:type" content="book" />
<meta property="og:url" content="https://moderndive.com/" />
<meta property="og:image" content="https://moderndive.com//images/logos/book_cover.png" />
<meta property="og:description" content="An open-source and fully-reproducible electronic textbook for teaching statistical inference using tidyverse data science tools." />
<meta name="github-repo" content="moderndive/ModernDive_book" />
<meta name="twitter:card" content="summary" />
<meta name="twitter:title" content="Chapter 3 Data Wrangling | Statistical Inference via Data Science" />
<meta name="twitter:site" content="@ModernDive" />
<meta name="twitter:description" content="An open-source and fully-reproducible electronic textbook for teaching statistical inference using tidyverse data science tools." />
<meta name="twitter:image" content="https://moderndive.com//images/logos/book_cover.png" />
<meta name="author" content="Chester Ismay and Albert Y. Kim Foreword by Kelly S. McConville Adapted by William R. Morgan" />
<meta name="date" content="2021-07-28" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<meta name="apple-mobile-web-app-capable" content="yes" />
<meta name="apple-mobile-web-app-status-bar-style" content="black" />
<link rel="apple-touch-icon-precomposed" sizes="152x152" href="images/logos/favicons/apple-touch-icon.png" />
<link rel="shortcut icon" href="images/logos/favicons/favicon.ico" type="image/x-icon" />
<link rel="prev" href="2-viz.html"/>
<link rel="next" href="4-tidy.html"/>
<script src="libs/header-attrs-2.9/header-attrs.js"></script>
<script src="libs/jquery-2.2.3/jquery.min.js"></script>
<link href="libs/gitbook-2.6.7/css/style.css" rel="stylesheet" />
<link href="libs/gitbook-2.6.7/css/plugin-table.css" rel="stylesheet" />
<link href="libs/gitbook-2.6.7/css/plugin-bookdown.css" rel="stylesheet" />
<link href="libs/gitbook-2.6.7/css/plugin-highlight.css" rel="stylesheet" />
<link href="libs/gitbook-2.6.7/css/plugin-search.css" rel="stylesheet" />
<link href="libs/gitbook-2.6.7/css/plugin-fontsettings.css" rel="stylesheet" />
<link href="libs/gitbook-2.6.7/css/plugin-clipboard.css" rel="stylesheet" />
<link href="libs/anchor-sections-1.0.1/anchor-sections.css" rel="stylesheet" />
<script src="libs/anchor-sections-1.0.1/anchor-sections.js"></script>
<script src="libs/kePrint-0.0.1/kePrint.js"></script>
<link href="libs/lightable-0.0.1/lightable.css" rel="stylesheet" />
<script src="libs/htmlwidgets-1.5.3/htmlwidgets.js"></script>
<link href="libs/dygraphs-1.1.1/dygraph.css" rel="stylesheet" />
<script src="libs/dygraphs-1.1.1/dygraph-combined.js"></script>
<script src="libs/dygraphs-1.1.1/shapes.js"></script>
<script src="libs/moment-2.8.4/moment.js"></script>
<script src="libs/moment-timezone-0.2.5/moment-timezone-with-data.js"></script>
<script src="libs/moment-fquarter-1.0.0/moment-fquarter.min.js"></script>
<script src="libs/dygraphs-binding-1.1.1.6/dygraphs.js"></script>
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-89938436-1', 'auto');
ga('send', 'pageview');
</script>
<style type="text/css">
pre > code.sourceCode { white-space: pre; position: relative; }
pre > code.sourceCode > span { display: inline-block; line-height: 1.25; }
pre > code.sourceCode > span:empty { height: 1.2em; }
.sourceCode { overflow: visible; }
code.sourceCode > span { color: inherit; text-decoration: inherit; }
pre.sourceCode { margin: 0; }
@media screen {
div.sourceCode { overflow: auto; }
}
@media print {
pre > code.sourceCode { white-space: pre-wrap; }
pre > code.sourceCode > span { text-indent: -5em; padding-left: 5em; }
}
pre.numberSource code
{ counter-reset: source-line 0; }
pre.numberSource code > span
{ position: relative; left: -4em; counter-increment: source-line; }
pre.numberSource code > span > a:first-child::before
{ content: counter(source-line);
position: relative; left: -1em; text-align: right; vertical-align: baseline;
border: none; display: inline-block;
-webkit-touch-callout: none; -webkit-user-select: none;
-khtml-user-select: none; -moz-user-select: none;
-ms-user-select: none; user-select: none;
padding: 0 4px; width: 4em;
color: #aaaaaa;
}
pre.numberSource { margin-left: 3em; border-left: 1px solid #aaaaaa; padding-left: 4px; }
div.sourceCode
{ }
@media screen {
pre > code.sourceCode > span > a:first-child::before { text-decoration: underline; }
}
code span.al { color: #ff0000; font-weight: bold; } /* Alert */
code span.an { color: #60a0b0; font-weight: bold; font-style: italic; } /* Annotation */
code span.at { color: #7d9029; } /* Attribute */
code span.bn { color: #40a070; } /* BaseN */
code span.bu { } /* BuiltIn */
code span.cf { color: #007020; font-weight: bold; } /* ControlFlow */
code span.ch { color: #4070a0; } /* Char */
code span.cn { color: #880000; } /* Constant */
code span.co { color: #60a0b0; font-style: italic; } /* Comment */
code span.cv { color: #60a0b0; font-weight: bold; font-style: italic; } /* CommentVar */
code span.do { color: #ba2121; font-style: italic; } /* Documentation */
code span.dt { color: #902000; } /* DataType */
code span.dv { color: #40a070; } /* DecVal */
code span.er { color: #ff0000; font-weight: bold; } /* Error */
code span.ex { } /* Extension */
code span.fl { color: #40a070; } /* Float */
code span.fu { color: #06287e; } /* Function */
code span.im { } /* Import */
code span.in { color: #60a0b0; font-weight: bold; font-style: italic; } /* Information */
code span.kw { color: #007020; font-weight: bold; } /* Keyword */
code span.op { color: #666666; } /* Operator */
code span.ot { color: #007020; } /* Other */
code span.pp { color: #bc7a00; } /* Preprocessor */
code span.sc { color: #4070a0; } /* SpecialChar */
code span.ss { color: #bb6688; } /* SpecialString */
code span.st { color: #4070a0; } /* String */
code span.va { color: #19177c; } /* Variable */
code span.vs { color: #4070a0; } /* VerbatimString */
code span.wa { color: #60a0b0; font-weight: bold; font-style: italic; } /* Warning */
</style>
<style type="text/css">
/* Used with Pandoc 2.11+ new --citeproc when CSL is used */
div.csl-bib-body { }
div.csl-entry {
clear: both;
}
.hanging div.csl-entry {
margin-left:2em;
text-indent:-2em;
}
div.csl-left-margin {
min-width:2em;
float:left;
}
div.csl-right-inline {
margin-left:2em;
padding-left:1em;
}
div.csl-indent {
margin-left: 2em;
}
</style>
<link rel="stylesheet" href="style.css" type="text/css" />
</head>
<body>
<div class="book without-animation with-summary font-size-2 font-family-1" data-basepath=".">
<div class="book-summary">
<nav role="navigation">
<ul class="summary">
<li class="chapter" data-level="" data-path="index.html"><a href="index.html"><i class="fa fa-check"></i>Welcome to ModernDive</a></li>
<li class="chapter" data-level="" data-path="foreword.html"><a href="foreword.html"><i class="fa fa-check"></i>Foreword</a></li>
<li class="chapter" data-level="" data-path="preface.html"><a href="preface.html"><i class="fa fa-check"></i>Preface</a>
<ul>
<li class="chapter" data-level="" data-path="preface.html"><a href="preface.html#introduction-for-students"><i class="fa fa-check"></i>Introduction for students</a>
<ul>
<li class="chapter" data-level="" data-path="preface.html"><a href="preface.html#what-we-hope-you-will-learn-from-this-book"><i class="fa fa-check"></i>What we hope you will learn from this book</a></li>
<li class="chapter" data-level="" data-path="preface.html"><a href="preface.html#datascience-pipeline"><i class="fa fa-check"></i>Data/science pipeline</a></li>
<li class="chapter" data-level="" data-path="preface.html"><a href="preface.html#reproducible-research"><i class="fa fa-check"></i>Reproducible research</a></li>
<li class="chapter" data-level="" data-path="preface.html"><a href="preface.html#final-note-for-students"><i class="fa fa-check"></i>Final note for students</a></li>
</ul></li>
<li class="chapter" data-level="" data-path="preface.html"><a href="preface.html#introduction-for-instructors"><i class="fa fa-check"></i>Introduction for instructors</a>
<ul>
<li class="chapter" data-level="" data-path="preface.html"><a href="preface.html#resources"><i class="fa fa-check"></i>Resources</a></li>
<li class="chapter" data-level="" data-path="preface.html"><a href="preface.html#why-did-we-write-this-book"><i class="fa fa-check"></i>Why did we write this book?</a></li>
<li class="chapter" data-level="" data-path="preface.html"><a href="preface.html#who-is-this-book-for"><i class="fa fa-check"></i>Who is this book for?</a></li>
</ul></li>
<li class="chapter" data-level="" data-path="preface.html"><a href="preface.html#connect-and-contribute"><i class="fa fa-check"></i>Connect and contribute</a></li>
<li class="chapter" data-level="" data-path="preface.html"><a href="preface.html#acknowledgements"><i class="fa fa-check"></i>Acknowledgements</a></li>
<li class="chapter" data-level="" data-path="preface.html"><a href="preface.html#about-this-book"><i class="fa fa-check"></i>About this book</a></li>
</ul></li>
<li class="chapter" data-level="" data-path="about-the-authors.html"><a href="about-the-authors.html"><i class="fa fa-check"></i>About the authors</a></li>
<li class="chapter" data-level="1" data-path="1-getting-started.html"><a href="1-getting-started.html"><i class="fa fa-check"></i><b>1</b> Getting Started with Data in R</a>
<ul>
<li class="chapter" data-level="1.1" data-path="1-getting-started.html"><a href="1-getting-started.html#r-rstudio"><i class="fa fa-check"></i><b>1.1</b> What are R and RStudio?</a>
<ul>
<li class="chapter" data-level="1.1.1" data-path="1-getting-started.html"><a href="1-getting-started.html#installing"><i class="fa fa-check"></i><b>1.1.1</b> Installing R and RStudio</a></li>
<li class="chapter" data-level="1.1.2" data-path="1-getting-started.html"><a href="1-getting-started.html#using-r-via-rstudio"><i class="fa fa-check"></i><b>1.1.2</b> Using R via RStudio</a></li>
</ul></li>
<li class="chapter" data-level="1.2" data-path="1-getting-started.html"><a href="1-getting-started.html#code"><i class="fa fa-check"></i><b>1.2</b> How do I code in R?</a>
<ul>
<li class="chapter" data-level="1.2.1" data-path="1-getting-started.html"><a href="1-getting-started.html#programming-concepts"><i class="fa fa-check"></i><b>1.2.1</b> Basic programming concepts and terminology</a></li>
<li class="chapter" data-level="1.2.2" data-path="1-getting-started.html"><a href="1-getting-started.html#messages"><i class="fa fa-check"></i><b>1.2.2</b> Errors, warnings, and messages</a></li>
<li class="chapter" data-level="1.2.3" data-path="1-getting-started.html"><a href="1-getting-started.html#tips-code"><i class="fa fa-check"></i><b>1.2.3</b> Tips on learning to code</a></li>
</ul></li>
<li class="chapter" data-level="1.3" data-path="1-getting-started.html"><a href="1-getting-started.html#packages"><i class="fa fa-check"></i><b>1.3</b> What are R packages?</a>
<ul>
<li class="chapter" data-level="1.3.1" data-path="1-getting-started.html"><a href="1-getting-started.html#package-installation"><i class="fa fa-check"></i><b>1.3.1</b> Package installation</a></li>
<li class="chapter" data-level="1.3.2" data-path="1-getting-started.html"><a href="1-getting-started.html#package-loading"><i class="fa fa-check"></i><b>1.3.2</b> Package loading</a></li>
<li class="chapter" data-level="1.3.3" data-path="1-getting-started.html"><a href="1-getting-started.html#package-use"><i class="fa fa-check"></i><b>1.3.3</b> Package use</a></li>
</ul></li>
<li class="chapter" data-level="1.4" data-path="1-getting-started.html"><a href="1-getting-started.html#rfishbase"><i class="fa fa-check"></i><b>1.4</b> Explore your first datasets</a>
<ul>
<li class="chapter" data-level="1.4.1" data-path="1-getting-started.html"><a href="1-getting-started.html#rfishpackage"><i class="fa fa-check"></i><b>1.4.1</b> <code>rfishbase</code> package</a></li>
<li class="chapter" data-level="1.4.2" data-path="1-getting-started.html"><a href="1-getting-started.html#fishbasedataframe"><i class="fa fa-check"></i><b>1.4.2</b> <code>fishbase</code> data frame</a></li>
<li class="chapter" data-level="1.4.3" data-path="1-getting-started.html"><a href="1-getting-started.html#exploredataframes"><i class="fa fa-check"></i><b>1.4.3</b> Exploring data frames</a></li>
<li class="chapter" data-level="1.4.4" data-path="1-getting-started.html"><a href="1-getting-started.html#identification-vs-measurement-variables"><i class="fa fa-check"></i><b>1.4.4</b> Identification and measurement variables</a></li>
<li class="chapter" data-level="1.4.5" data-path="1-getting-started.html"><a href="1-getting-started.html#help-files"><i class="fa fa-check"></i><b>1.4.5</b> Help files</a></li>
</ul></li>
<li class="chapter" data-level="1.5" data-path="1-getting-started.html"><a href="1-getting-started.html#conclusion"><i class="fa fa-check"></i><b>1.5</b> Conclusion</a>
<ul>
<li class="chapter" data-level="1.5.1" data-path="1-getting-started.html"><a href="1-getting-started.html#additional-resources"><i class="fa fa-check"></i><b>1.5.1</b> Additional resources</a></li>
<li class="chapter" data-level="1.5.2" data-path="1-getting-started.html"><a href="1-getting-started.html#whats-to-come"><i class="fa fa-check"></i><b>1.5.2</b> What’s to come?</a></li>
</ul></li>
</ul></li>
<li class="part"><span><b>I Data Science with tidyverse</b></span></li>
<li class="chapter" data-level="2" data-path="2-viz.html"><a href="2-viz.html"><i class="fa fa-check"></i><b>2</b> Data Visualization</a>
<ul>
<li class="chapter" data-level="" data-path="2-viz.html"><a href="2-viz.html#needed-packages"><i class="fa fa-check"></i>Needed packages</a></li>
<li class="chapter" data-level="2.1" data-path="2-viz.html"><a href="2-viz.html#grammarofgraphics"><i class="fa fa-check"></i><b>2.1</b> The grammar of graphics</a>
<ul>
<li class="chapter" data-level="2.1.1" data-path="2-viz.html"><a href="2-viz.html#components-of-the-grammar"><i class="fa fa-check"></i><b>2.1.1</b> Components of the grammar</a></li>
<li class="chapter" data-level="2.1.2" data-path="2-viz.html"><a href="2-viz.html#gapminder"><i class="fa fa-check"></i><b>2.1.2</b> Gapminder data</a></li>
<li class="chapter" data-level="2.1.3" data-path="2-viz.html"><a href="2-viz.html#other-components"><i class="fa fa-check"></i><b>2.1.3</b> Other components</a></li>
<li class="chapter" data-level="2.1.4" data-path="2-viz.html"><a href="2-viz.html#ggplot2-package"><i class="fa fa-check"></i><b>2.1.4</b> ggplot2 package</a></li>
</ul></li>
<li class="chapter" data-level="2.2" data-path="2-viz.html"><a href="2-viz.html#FiveNG"><i class="fa fa-check"></i><b>2.2</b> Five named graphs - the 5NG</a></li>
<li class="chapter" data-level="2.3" data-path="2-viz.html"><a href="2-viz.html#scatterplots"><i class="fa fa-check"></i><b>2.3</b> 5NG#1: Scatterplots</a>
<ul>
<li class="chapter" data-level="2.3.1" data-path="2-viz.html"><a href="2-viz.html#geompoint"><i class="fa fa-check"></i><b>2.3.1</b> Scatterplots via <code>geom_point</code></a></li>
<li class="chapter" data-level="2.3.2" data-path="2-viz.html"><a href="2-viz.html#overplotting"><i class="fa fa-check"></i><b>2.3.2</b> Overplotting</a></li>
<li class="chapter" data-level="2.3.3" data-path="2-viz.html"><a href="2-viz.html#summary"><i class="fa fa-check"></i><b>2.3.3</b> Summary</a></li>
</ul></li>
<li class="chapter" data-level="2.4" data-path="2-viz.html"><a href="2-viz.html#linegraphs"><i class="fa fa-check"></i><b>2.4</b> 5NG#2: Linegraphs</a>
<ul>
<li class="chapter" data-level="2.4.1" data-path="2-viz.html"><a href="2-viz.html#geomline"><i class="fa fa-check"></i><b>2.4.1</b> Linegraphs via <code>geom_line</code></a></li>
<li class="chapter" data-level="2.4.2" data-path="2-viz.html"><a href="2-viz.html#summary-1"><i class="fa fa-check"></i><b>2.4.2</b> Summary</a></li>
</ul></li>
<li class="chapter" data-level="2.5" data-path="2-viz.html"><a href="2-viz.html#facets"><i class="fa fa-check"></i><b>2.5</b> Facets</a></li>
<li class="chapter" data-level="2.6" data-path="2-viz.html"><a href="2-viz.html#histograms"><i class="fa fa-check"></i><b>2.6</b> 5NG#3: Histograms</a>
<ul>
<li class="chapter" data-level="2.6.1" data-path="2-viz.html"><a href="2-viz.html#geomhistogram"><i class="fa fa-check"></i><b>2.6.1</b> Histograms via <code>geom_histogram</code></a></li>
<li class="chapter" data-level="2.6.2" data-path="2-viz.html"><a href="2-viz.html#adjustbins"><i class="fa fa-check"></i><b>2.6.2</b> Adjusting the bins</a></li>
<li class="chapter" data-level="2.6.3" data-path="2-viz.html"><a href="2-viz.html#summary-2"><i class="fa fa-check"></i><b>2.6.3</b> Summary</a></li>
</ul></li>
<li class="chapter" data-level="2.7" data-path="2-viz.html"><a href="2-viz.html#boxplots"><i class="fa fa-check"></i><b>2.7</b> 5NG#4: Boxplots</a>
<ul>
<li class="chapter" data-level="2.7.1" data-path="2-viz.html"><a href="2-viz.html#geomboxplot"><i class="fa fa-check"></i><b>2.7.1</b> Boxplots via <code>geom_boxplot</code></a></li>
<li class="chapter" data-level="2.7.2" data-path="2-viz.html"><a href="2-viz.html#summary-3"><i class="fa fa-check"></i><b>2.7.2</b> Summary</a></li>
</ul></li>
<li class="chapter" data-level="2.8" data-path="2-viz.html"><a href="2-viz.html#geombar"><i class="fa fa-check"></i><b>2.8</b> 5NG#5: Barplots</a>
<ul>
<li class="chapter" data-level="2.8.1" data-path="2-viz.html"><a href="2-viz.html#barplots-via-geom_bar-or-geom_col"><i class="fa fa-check"></i><b>2.8.1</b> Barplots via <code>geom_bar</code> or <code>geom_col</code></a></li>
<li class="chapter" data-level="2.8.2" data-path="2-viz.html"><a href="2-viz.html#must-avoid-pie-charts"><i class="fa fa-check"></i><b>2.8.2</b> Must avoid pie charts!</a></li>
<li class="chapter" data-level="2.8.3" data-path="2-viz.html"><a href="2-viz.html#two-categ-barplot"><i class="fa fa-check"></i><b>2.8.3</b> Two categorical variables</a></li>
<li class="chapter" data-level="2.8.4" data-path="2-viz.html"><a href="2-viz.html#summary-4"><i class="fa fa-check"></i><b>2.8.4</b> Summary</a></li>
</ul></li>
<li class="chapter" data-level="2.9" data-path="2-viz.html"><a href="2-viz.html#data-vis-conclusion"><i class="fa fa-check"></i><b>2.9</b> Conclusion</a>
<ul>
<li class="chapter" data-level="2.9.1" data-path="2-viz.html"><a href="2-viz.html#summary-table"><i class="fa fa-check"></i><b>2.9.1</b> Summary table</a></li>
<li class="chapter" data-level="2.9.2" data-path="2-viz.html"><a href="2-viz.html#function-argument-specification"><i class="fa fa-check"></i><b>2.9.2</b> Function argument specification</a></li>
<li class="chapter" data-level="2.9.3" data-path="2-viz.html"><a href="2-viz.html#additional-resources-1"><i class="fa fa-check"></i><b>2.9.3</b> Additional resources</a></li>
<li class="chapter" data-level="2.9.4" data-path="2-viz.html"><a href="2-viz.html#whats-to-come-3"><i class="fa fa-check"></i><b>2.9.4</b> What’s to come</a></li>
</ul></li>
</ul></li>
<li class="chapter" data-level="3" data-path="3-wrangling.html"><a href="3-wrangling.html"><i class="fa fa-check"></i><b>3</b> Data Wrangling</a>
<ul>
<li class="chapter" data-level="" data-path="3-wrangling.html"><a href="3-wrangling.html#wrangling-packages"><i class="fa fa-check"></i>Needed packages</a></li>
<li class="chapter" data-level="3.1" data-path="3-wrangling.html"><a href="3-wrangling.html#piping"><i class="fa fa-check"></i><b>3.1</b> The pipe operator: <code>%>%</code></a></li>
<li class="chapter" data-level="3.2" data-path="3-wrangling.html"><a href="3-wrangling.html#filter"><i class="fa fa-check"></i><b>3.2</b> <code>filter</code> rows</a></li>
<li class="chapter" data-level="3.3" data-path="3-wrangling.html"><a href="3-wrangling.html#slice-rows"><i class="fa fa-check"></i><b>3.3</b> <code>slice</code> rows</a></li>
<li class="chapter" data-level="3.4" data-path="3-wrangling.html"><a href="3-wrangling.html#select"><i class="fa fa-check"></i><b>3.4</b> <code>select</code> variables</a>
<ul>
<li class="chapter" data-level="3.4.1" data-path="3-wrangling.html"><a href="3-wrangling.html#rename"><i class="fa fa-check"></i><b>3.4.1</b> <code>rename</code> variables</a></li>
</ul></li>
<li class="chapter" data-level="3.5" data-path="3-wrangling.html"><a href="3-wrangling.html#summarize"><i class="fa fa-check"></i><b>3.5</b> <code>summarize</code> variables</a></li>
<li class="chapter" data-level="3.6" data-path="3-wrangling.html"><a href="3-wrangling.html#groupby"><i class="fa fa-check"></i><b>3.6</b> <code>group_by</code> rows</a>
<ul>
<li class="chapter" data-level="3.6.1" data-path="3-wrangling.html"><a href="3-wrangling.html#grouping-by-more-than-one-variable"><i class="fa fa-check"></i><b>3.6.1</b> Grouping by more than one variable</a></li>
</ul></li>
<li class="chapter" data-level="3.7" data-path="3-wrangling.html"><a href="3-wrangling.html#mutate"><i class="fa fa-check"></i><b>3.7</b> <code>mutate</code> existing variables</a></li>
<li class="chapter" data-level="3.8" data-path="3-wrangling.html"><a href="3-wrangling.html#arrange"><i class="fa fa-check"></i><b>3.8</b> <code>arrange</code> and sort rows</a></li>
<li class="chapter" data-level="3.9" data-path="3-wrangling.html"><a href="3-wrangling.html#joins"><i class="fa fa-check"></i><b>3.9</b> <code>join</code> data frames</a></li>
<li class="chapter" data-level="3.10" data-path="3-wrangling.html"><a href="3-wrangling.html#wrangling-conclusion"><i class="fa fa-check"></i><b>3.10</b> Conclusion</a>
<ul>
<li class="chapter" data-level="3.10.1" data-path="3-wrangling.html"><a href="3-wrangling.html#summary-table-1"><i class="fa fa-check"></i><b>3.10.1</b> Summary table</a></li>
<li class="chapter" data-level="3.10.2" data-path="3-wrangling.html"><a href="3-wrangling.html#additional-resources-2"><i class="fa fa-check"></i><b>3.10.2</b> Additional resources</a></li>
<li class="chapter" data-level="3.10.3" data-path="3-wrangling.html"><a href="3-wrangling.html#whats-to-come-1"><i class="fa fa-check"></i><b>3.10.3</b> What’s to come?</a></li>
</ul></li>
</ul></li>
<li class="chapter" data-level="4" data-path="4-tidy.html"><a href="4-tidy.html"><i class="fa fa-check"></i><b>4</b> Data Importing and “Tidy” Data</a>
<ul>
<li class="chapter" data-level="" data-path="4-tidy.html"><a href="4-tidy.html#tidy-packages"><i class="fa fa-check"></i>Needed packages</a></li>
<li class="chapter" data-level="4.1" data-path="4-tidy.html"><a href="4-tidy.html#csv"><i class="fa fa-check"></i><b>4.1</b> Importing data</a>
<ul>
<li class="chapter" data-level="4.1.1" data-path="4-tidy.html"><a href="4-tidy.html#using-the-console"><i class="fa fa-check"></i><b>4.1.1</b> Using the console</a></li>
<li class="chapter" data-level="4.1.2" data-path="4-tidy.html"><a href="4-tidy.html#using-rstudios-interface"><i class="fa fa-check"></i><b>4.1.2</b> Using RStudio’s interface</a></li>
</ul></li>
<li class="chapter" data-level="4.2" data-path="4-tidy.html"><a href="4-tidy.html#tidy-data-ex"><i class="fa fa-check"></i><b>4.2</b> “Tidy” data</a>
<ul>
<li class="chapter" data-level="4.2.1" data-path="4-tidy.html"><a href="4-tidy.html#tidy-definition"><i class="fa fa-check"></i><b>4.2.1</b> Definition of “tidy” data</a></li>
<li class="chapter" data-level="4.2.2" data-path="4-tidy.html"><a href="4-tidy.html#converting-to-tidy-data"><i class="fa fa-check"></i><b>4.2.2</b> Converting to “tidy” data</a></li>
</ul></li>
<li class="chapter" data-level="4.3" data-path="4-tidy.html"><a href="4-tidy.html#case-study-tidy"><i class="fa fa-check"></i><b>4.3</b> Case study: Weight loss data</a></li>
<li class="chapter" data-level="4.4" data-path="4-tidy.html"><a href="4-tidy.html#tidyverse-package"><i class="fa fa-check"></i><b>4.4</b> <code>tidyverse</code> package</a></li>
<li class="chapter" data-level="4.5" data-path="4-tidy.html"><a href="4-tidy.html#tidy-data-conclusion"><i class="fa fa-check"></i><b>4.5</b> Conclusion</a>
<ul>
<li class="chapter" data-level="4.5.1" data-path="4-tidy.html"><a href="4-tidy.html#additional-resources-3"><i class="fa fa-check"></i><b>4.5.1</b> Additional resources</a></li>
<li class="chapter" data-level="4.5.2" data-path="4-tidy.html"><a href="4-tidy.html#whats-to-come-2"><i class="fa fa-check"></i><b>4.5.2</b> What’s to come?</a></li>
</ul></li>
</ul></li>
<li class="part"><span><b>II Data Modeling with moderndive</b></span></li>
<li class="chapter" data-level="5" data-path="5-regression.html"><a href="5-regression.html"><i class="fa fa-check"></i><b>5</b> Basic Regression</a>
<ul>
<li class="chapter" data-level="" data-path="5-regression.html"><a href="5-regression.html#reg-packages"><i class="fa fa-check"></i>Needed packages</a></li>
<li class="chapter" data-level="5.1" data-path="5-regression.html"><a href="5-regression.html#model1"><i class="fa fa-check"></i><b>5.1</b> One numerical explanatory variable</a>
<ul>
<li class="chapter" data-level="5.1.1" data-path="5-regression.html"><a href="5-regression.html#model1EDA"><i class="fa fa-check"></i><b>5.1.1</b> Exploratory data analysis</a></li>
<li class="chapter" data-level="5.1.2" data-path="5-regression.html"><a href="5-regression.html#model1table"><i class="fa fa-check"></i><b>5.1.2</b> Simple linear regression</a></li>
<li class="chapter" data-level="5.1.3" data-path="5-regression.html"><a href="5-regression.html#model1points"><i class="fa fa-check"></i><b>5.1.3</b> Observed/fitted values and residuals</a></li>
</ul></li>
<li class="chapter" data-level="5.2" data-path="5-regression.html"><a href="5-regression.html#model2"><i class="fa fa-check"></i><b>5.2</b> One categorical explanatory variable</a>
<ul>
<li class="chapter" data-level="5.2.1" data-path="5-regression.html"><a href="5-regression.html#model2EDA"><i class="fa fa-check"></i><b>5.2.1</b> Exploratory data analysis</a></li>
<li class="chapter" data-level="5.2.2" data-path="5-regression.html"><a href="5-regression.html#model2table"><i class="fa fa-check"></i><b>5.2.2</b> Linear regression</a></li>
<li class="chapter" data-level="5.2.3" data-path="5-regression.html"><a href="5-regression.html#model2points"><i class="fa fa-check"></i><b>5.2.3</b> Observed/fitted values and residuals</a></li>
</ul></li>
<li class="chapter" data-level="5.3" data-path="5-regression.html"><a href="5-regression.html#reg-related-topics"><i class="fa fa-check"></i><b>5.3</b> Related topics</a>
<ul>
<li class="chapter" data-level="5.3.1" data-path="5-regression.html"><a href="5-regression.html#correlation-is-not-causation"><i class="fa fa-check"></i><b>5.3.1</b> Correlation is not necessarily causation</a></li>
<li class="chapter" data-level="5.3.2" data-path="5-regression.html"><a href="5-regression.html#leastsquares"><i class="fa fa-check"></i><b>5.3.2</b> Best-fitting line</a></li>
<li class="chapter" data-level="5.3.3" data-path="5-regression.html"><a href="5-regression.html#underthehood"><i class="fa fa-check"></i><b>5.3.3</b> <code>get_regression_x()</code> functions</a></li>
</ul></li>
<li class="chapter" data-level="5.4" data-path="5-regression.html"><a href="5-regression.html#reg-conclusion"><i class="fa fa-check"></i><b>5.4</b> Conclusion</a>
<ul>
<li class="chapter" data-level="5.4.1" data-path="5-regression.html"><a href="5-regression.html#additional-resources-basic-regression"><i class="fa fa-check"></i><b>5.4.1</b> Additional resources</a></li>
<li class="chapter" data-level="5.4.2" data-path="5-regression.html"><a href="5-regression.html#whats-to-come-4"><i class="fa fa-check"></i><b>5.4.2</b> What’s to come?</a></li>
</ul></li>
</ul></li>
<li class="chapter" data-level="6" data-path="6-multiple-regression.html"><a href="6-multiple-regression.html"><i class="fa fa-check"></i><b>6</b> Multiple Regression</a>
<ul>
<li class="chapter" data-level="" data-path="6-multiple-regression.html"><a href="6-multiple-regression.html#mult-reg-packages"><i class="fa fa-check"></i>Needed packages</a></li>
<li class="chapter" data-level="6.1" data-path="6-multiple-regression.html"><a href="6-multiple-regression.html#model4"><i class="fa fa-check"></i><b>6.1</b> One numerical and one categorical explanatory variable</a>
<ul>
<li class="chapter" data-level="6.1.1" data-path="6-multiple-regression.html"><a href="6-multiple-regression.html#model4EDA"><i class="fa fa-check"></i><b>6.1.1</b> Exploratory data analysis</a></li>
<li class="chapter" data-level="6.1.2" data-path="6-multiple-regression.html"><a href="6-multiple-regression.html#model4interactiontable"><i class="fa fa-check"></i><b>6.1.2</b> Interaction model</a></li>
<li class="chapter" data-level="6.1.3" data-path="6-multiple-regression.html"><a href="6-multiple-regression.html#model4table"><i class="fa fa-check"></i><b>6.1.3</b> Parallel slopes model</a></li>
<li class="chapter" data-level="6.1.4" data-path="6-multiple-regression.html"><a href="6-multiple-regression.html#model4points"><i class="fa fa-check"></i><b>6.1.4</b> Observed/fitted values and residuals</a></li>
</ul></li>
<li class="chapter" data-level="6.2" data-path="6-multiple-regression.html"><a href="6-multiple-regression.html#model3"><i class="fa fa-check"></i><b>6.2</b> Two categorical explanatory variables</a>
<ul>
<li class="chapter" data-level="6.2.1" data-path="6-multiple-regression.html"><a href="6-multiple-regression.html#model3EDA"><i class="fa fa-check"></i><b>6.2.1</b> Exploratory data analysis</a></li>
<li class="chapter" data-level="6.2.2" data-path="6-multiple-regression.html"><a href="6-multiple-regression.html#model3table"><i class="fa fa-check"></i><b>6.2.2</b> Regression lines</a></li>
<li class="chapter" data-level="6.2.3" data-path="6-multiple-regression.html"><a href="6-multiple-regression.html#model3points"><i class="fa fa-check"></i><b>6.2.3</b> Observed/fitted values and residuals</a></li>
</ul></li>
<li class="chapter" data-level="6.3" data-path="6-multiple-regression.html"><a href="6-multiple-regression.html#mult-reg-related-topics"><i class="fa fa-check"></i><b>6.3</b> Related topics</a>
<ul>
<li class="chapter" data-level="6.3.1" data-path="6-multiple-regression.html"><a href="6-multiple-regression.html#model-selection"><i class="fa fa-check"></i><b>6.3.1</b> Model selection using visualizations</a></li>
<li class="chapter" data-level="6.3.2" data-path="6-multiple-regression.html"><a href="6-multiple-regression.html#rsquared"><i class="fa fa-check"></i><b>6.3.2</b> Model selection using R-squared</a></li>
</ul></li>
<li class="chapter" data-level="6.4" data-path="6-multiple-regression.html"><a href="6-multiple-regression.html#mult-reg-conclusion"><i class="fa fa-check"></i><b>6.4</b> Conclusion</a>
<ul>
<li class="chapter" data-level="6.4.1" data-path="6-multiple-regression.html"><a href="6-multiple-regression.html#additional-resources-4"><i class="fa fa-check"></i><b>6.4.1</b> Additional resources</a></li>
<li class="chapter" data-level="6.4.2" data-path="6-multiple-regression.html"><a href="6-multiple-regression.html#whats-to-come-5"><i class="fa fa-check"></i><b>6.4.2</b> What’s to come?</a></li>
</ul></li>
</ul></li>
<li class="part"><span><b>III Statistical Inference with infer</b></span></li>
<li class="chapter" data-level="7" data-path="7-sampling.html"><a href="7-sampling.html"><i class="fa fa-check"></i><b>7</b> Sampling</a>
<ul>
<li class="chapter" data-level="" data-path="7-sampling.html"><a href="7-sampling.html#sampling-packages"><i class="fa fa-check"></i>Needed packages</a></li>
<li class="chapter" data-level="7.1" data-path="7-sampling.html"><a href="7-sampling.html#sampling-activity"><i class="fa fa-check"></i><b>7.1</b> Sampling bowl activity</a>
<ul>
<li class="chapter" data-level="7.1.1" data-path="7-sampling.html"><a href="7-sampling.html#what-proportion-of-this-bowls-balls-are-red"><i class="fa fa-check"></i><b>7.1.1</b> What proportion of this bowl’s balls are red?</a></li>
<li class="chapter" data-level="7.1.2" data-path="7-sampling.html"><a href="7-sampling.html#using-the-shovel-once"><i class="fa fa-check"></i><b>7.1.2</b> Using the shovel once</a></li>
<li class="chapter" data-level="7.1.3" data-path="7-sampling.html"><a href="7-sampling.html#student-shovels"><i class="fa fa-check"></i><b>7.1.3</b> Using the shovel 33 times</a></li>
<li class="chapter" data-level="7.1.4" data-path="7-sampling.html"><a href="7-sampling.html#sampling-what-did-we-just-do"><i class="fa fa-check"></i><b>7.1.4</b> What did we just do?</a></li>
</ul></li>
<li class="chapter" data-level="7.2" data-path="7-sampling.html"><a href="7-sampling.html#sampling-simulation"><i class="fa fa-check"></i><b>7.2</b> Virtual sampling</a>
<ul>
<li class="chapter" data-level="7.2.1" data-path="7-sampling.html"><a href="7-sampling.html#using-the-virtual-shovel-once"><i class="fa fa-check"></i><b>7.2.1</b> Using the virtual shovel once</a></li>
</ul></li>
<li class="chapter" data-level="7.3" data-path="7-sampling.html"><a href="7-sampling.html#sampling-framework"><i class="fa fa-check"></i><b>7.3</b> Sampling framework</a>
<ul>
<li class="chapter" data-level="7.3.1" data-path="7-sampling.html"><a href="7-sampling.html#terminology-and-notation"><i class="fa fa-check"></i><b>7.3.1</b> Terminology and notation</a></li>
<li class="chapter" data-level="7.3.2" data-path="7-sampling.html"><a href="7-sampling.html#sampling-definitions"><i class="fa fa-check"></i><b>7.3.2</b> Statistical definitions</a></li>
<li class="chapter" data-level="7.3.3" data-path="7-sampling.html"><a href="7-sampling.html#moral-of-the-story"><i class="fa fa-check"></i><b>7.3.3</b> The moral of the story</a></li>
</ul></li>
<li class="chapter" data-level="7.4" data-path="7-sampling.html"><a href="7-sampling.html#sampling-case-study"><i class="fa fa-check"></i><b>7.4</b> Case study: Polls</a></li>
<li class="chapter" data-level="7.5" data-path="7-sampling.html"><a href="7-sampling.html#sampling-conclusion-central-limit-theorem"><i class="fa fa-check"></i><b>7.5</b> Central Limit Theorem</a></li>
<li class="chapter" data-level="7.6" data-path="7-sampling.html"><a href="7-sampling.html#sampling-conclusion"><i class="fa fa-check"></i><b>7.6</b> Conclusion</a>
<ul>
<li class="chapter" data-level="7.6.1" data-path="7-sampling.html"><a href="7-sampling.html#sampling-conclusion-table"><i class="fa fa-check"></i><b>7.6.1</b> Sampling scenarios</a></li>
<li class="chapter" data-level="7.6.2" data-path="7-sampling.html"><a href="7-sampling.html#additional-resources-5"><i class="fa fa-check"></i><b>7.6.2</b> Additional resources</a></li>
<li class="chapter" data-level="7.6.3" data-path="7-sampling.html"><a href="7-sampling.html#whats-to-come-6"><i class="fa fa-check"></i><b>7.6.3</b> What’s to come?</a></li>
</ul></li>
</ul></li>
<li class="chapter" data-level="8" data-path="8-confidence-intervals.html"><a href="8-confidence-intervals.html"><i class="fa fa-check"></i><b>8</b> Bootstrapping and Confidence Intervals</a>
<ul>
<li class="chapter" data-level="" data-path="8-confidence-intervals.html"><a href="8-confidence-intervals.html#CI-packages"><i class="fa fa-check"></i>Needed packages</a></li>
<li class="chapter" data-level="8.1" data-path="8-confidence-intervals.html"><a href="8-confidence-intervals.html#resampling-tactile"><i class="fa fa-check"></i><b>8.1</b> Pennies activity</a>
<ul>
<li class="chapter" data-level="8.1.1" data-path="8-confidence-intervals.html"><a href="8-confidence-intervals.html#what-is-the-average-year-on-us-pennies-in-2019"><i class="fa fa-check"></i><b>8.1.1</b> What is the average year on US pennies in 2019?</a></li>
<li class="chapter" data-level="8.1.2" data-path="8-confidence-intervals.html"><a href="8-confidence-intervals.html#resampling-once"><i class="fa fa-check"></i><b>8.1.2</b> Resampling once</a></li>
<li class="chapter" data-level="8.1.3" data-path="8-confidence-intervals.html"><a href="8-confidence-intervals.html#student-resamples"><i class="fa fa-check"></i><b>8.1.3</b> Resampling 35 times</a></li>
<li class="chapter" data-level="8.1.4" data-path="8-confidence-intervals.html"><a href="8-confidence-intervals.html#ci-what-did-we-just-do"><i class="fa fa-check"></i><b>8.1.4</b> What did we just do?</a></li>
</ul></li>
<li class="chapter" data-level="8.2" data-path="8-confidence-intervals.html"><a href="8-confidence-intervals.html#resampling-simulation"><i class="fa fa-check"></i><b>8.2</b> Computer simulation of resampling</a>
<ul>
<li class="chapter" data-level="8.2.1" data-path="8-confidence-intervals.html"><a href="8-confidence-intervals.html#virtually-resampling-once"><i class="fa fa-check"></i><b>8.2.1</b> Virtually resampling once</a></li>
<li class="chapter" data-level="8.2.2" data-path="8-confidence-intervals.html"><a href="8-confidence-intervals.html#bootstrap-35-replicates"><i class="fa fa-check"></i><b>8.2.2</b> Virtually resampling 35 times</a></li>
<li class="chapter" data-level="8.2.3" data-path="8-confidence-intervals.html"><a href="8-confidence-intervals.html#bootstrap-1000-replicates"><i class="fa fa-check"></i><b>8.2.3</b> Virtually resampling 1000 times</a></li>
</ul></li>
<li class="chapter" data-level="8.3" data-path="8-confidence-intervals.html"><a href="8-confidence-intervals.html#ci-build-up"><i class="fa fa-check"></i><b>8.3</b> Understanding confidence intervals</a>
<ul>
<li class="chapter" data-level="8.3.1" data-path="8-confidence-intervals.html"><a href="8-confidence-intervals.html#percentile-method"><i class="fa fa-check"></i><b>8.3.1</b> Percentile method</a></li>
<li class="chapter" data-level="8.3.2" data-path="8-confidence-intervals.html"><a href="8-confidence-intervals.html#se-method"><i class="fa fa-check"></i><b>8.3.2</b> Standard error method</a></li>
</ul></li>
<li class="chapter" data-level="8.4" data-path="8-confidence-intervals.html"><a href="8-confidence-intervals.html#bootstrap-process"><i class="fa fa-check"></i><b>8.4</b> Constructing confidence intervals</a>
<ul>
<li class="chapter" data-level="8.4.1" data-path="8-confidence-intervals.html"><a href="8-confidence-intervals.html#original-workflow"><i class="fa fa-check"></i><b>8.4.1</b> Original workflow</a></li>
<li class="chapter" data-level="8.4.2" data-path="8-confidence-intervals.html"><a href="8-confidence-intervals.html#infer-workflow"><i class="fa fa-check"></i><b>8.4.2</b> <code>infer</code> package workflow</a></li>
<li class="chapter" data-level="8.4.3" data-path="8-confidence-intervals.html"><a href="8-confidence-intervals.html#percentile-method-infer"><i class="fa fa-check"></i><b>8.4.3</b> Percentile method with <code>infer</code></a></li>
<li class="chapter" data-level="8.4.4" data-path="8-confidence-intervals.html"><a href="8-confidence-intervals.html#infer-se"><i class="fa fa-check"></i><b>8.4.4</b> Standard error method with <code>infer</code></a></li>
</ul></li>
<li class="chapter" data-level="8.5" data-path="8-confidence-intervals.html"><a href="8-confidence-intervals.html#one-prop-ci"><i class="fa fa-check"></i><b>8.5</b> Interpreting confidence intervals</a>
<ul>
<li class="chapter" data-level="8.5.1" data-path="8-confidence-intervals.html"><a href="8-confidence-intervals.html#ilyas-yohan"><i class="fa fa-check"></i><b>8.5.1</b> Did the net capture the fish?</a></li>
<li class="chapter" data-level="8.5.2" data-path="8-confidence-intervals.html"><a href="8-confidence-intervals.html#shorthand"><i class="fa fa-check"></i><b>8.5.2</b> Precise and shorthand interpretation</a></li>
<li class="chapter" data-level="8.5.3" data-path="8-confidence-intervals.html"><a href="8-confidence-intervals.html#ci-width"><i class="fa fa-check"></i><b>8.5.3</b> Width of confidence intervals</a></li>
</ul></li>
<li class="chapter" data-level="8.6" data-path="8-confidence-intervals.html"><a href="8-confidence-intervals.html#case-study-two-prop-ci"><i class="fa fa-check"></i><b>8.6</b> Case study: Is yawning contagious?</a>
<ul>
<li class="chapter" data-level="8.6.1" data-path="8-confidence-intervals.html"><a href="8-confidence-intervals.html#mythbusters-study-data"><i class="fa fa-check"></i><b>8.6.1</b> <em>Mythbusters</em> study data</a></li>
<li class="chapter" data-level="8.6.2" data-path="8-confidence-intervals.html"><a href="8-confidence-intervals.html#sampling-scenario"><i class="fa fa-check"></i><b>8.6.2</b> Sampling scenario</a></li>
<li class="chapter" data-level="8.6.3" data-path="8-confidence-intervals.html"><a href="8-confidence-intervals.html#ci-build"><i class="fa fa-check"></i><b>8.6.3</b> Constructing the confidence interval</a></li>
<li class="chapter" data-level="8.6.4" data-path="8-confidence-intervals.html"><a href="8-confidence-intervals.html#interpreting-the-confidence-interval"><i class="fa fa-check"></i><b>8.6.4</b> Interpreting the confidence interval</a></li>
</ul></li>
<li class="chapter" data-level="8.7" data-path="8-confidence-intervals.html"><a href="8-confidence-intervals.html#ci-conclusion"><i class="fa fa-check"></i><b>8.7</b> Conclusion</a>
<ul>
<li class="chapter" data-level="8.7.1" data-path="8-confidence-intervals.html"><a href="8-confidence-intervals.html#bootstrap-vs-sampling"><i class="fa fa-check"></i><b>8.7.1</b> Comparing bootstrap and sampling distributions</a></li>
<li class="chapter" data-level="8.7.2" data-path="8-confidence-intervals.html"><a href="8-confidence-intervals.html#theory-ci"><i class="fa fa-check"></i><b>8.7.2</b> Theory-based confidence intervals</a></li>
<li class="chapter" data-level="8.7.3" data-path="8-confidence-intervals.html"><a href="8-confidence-intervals.html#additional-resources-6"><i class="fa fa-check"></i><b>8.7.3</b> Additional resources</a></li>
<li class="chapter" data-level="8.7.4" data-path="8-confidence-intervals.html"><a href="8-confidence-intervals.html#whats-to-come-7"><i class="fa fa-check"></i><b>8.7.4</b> What’s to come?</a></li>
</ul></li>
</ul></li>
<li class="chapter" data-level="9" data-path="9-hypothesis-testing.html"><a href="9-hypothesis-testing.html"><i class="fa fa-check"></i><b>9</b> Hypothesis Testing</a>
<ul>
<li class="chapter" data-level="" data-path="9-hypothesis-testing.html"><a href="9-hypothesis-testing.html#nhst-packages"><i class="fa fa-check"></i>Needed packages</a></li>
<li class="chapter" data-level="9.1" data-path="9-hypothesis-testing.html"><a href="9-hypothesis-testing.html#ht-activity"><i class="fa fa-check"></i><b>9.1</b> Promotions activity</a>
<ul>
<li class="chapter" data-level="9.1.1" data-path="9-hypothesis-testing.html"><a href="9-hypothesis-testing.html#does-gender-affect-promotions-at-a-bank"><i class="fa fa-check"></i><b>9.1.1</b> Does gender affect promotions at a bank?</a></li>
<li class="chapter" data-level="9.1.2" data-path="9-hypothesis-testing.html"><a href="9-hypothesis-testing.html#shuffling-once"><i class="fa fa-check"></i><b>9.1.2</b> Shuffling once</a></li>
<li class="chapter" data-level="9.1.3" data-path="9-hypothesis-testing.html"><a href="9-hypothesis-testing.html#shuffling-16-times"><i class="fa fa-check"></i><b>9.1.3</b> Shuffling 16 times</a></li>
<li class="chapter" data-level="9.1.4" data-path="9-hypothesis-testing.html"><a href="9-hypothesis-testing.html#ht-what-did-we-just-do"><i class="fa fa-check"></i><b>9.1.4</b> What did we just do?</a></li>
</ul></li>
<li class="chapter" data-level="9.2" data-path="9-hypothesis-testing.html"><a href="9-hypothesis-testing.html#understanding-ht"><i class="fa fa-check"></i><b>9.2</b> Understanding hypothesis tests</a></li>
<li class="chapter" data-level="9.3" data-path="9-hypothesis-testing.html"><a href="9-hypothesis-testing.html#ht-infer"><i class="fa fa-check"></i><b>9.3</b> Conducting hypothesis tests</a>
<ul>
<li class="chapter" data-level="9.3.1" data-path="9-hypothesis-testing.html"><a href="9-hypothesis-testing.html#infer-workflow-ht"><i class="fa fa-check"></i><b>9.3.1</b> <code>infer</code> package workflow</a></li>
<li class="chapter" data-level="9.3.2" data-path="9-hypothesis-testing.html"><a href="9-hypothesis-testing.html#comparing-infer-workflows"><i class="fa fa-check"></i><b>9.3.2</b> Comparison with confidence intervals</a></li>
<li class="chapter" data-level="9.3.3" data-path="9-hypothesis-testing.html"><a href="9-hypothesis-testing.html#only-one-test"><i class="fa fa-check"></i><b>9.3.3</b> “There is only one test”</a></li>
</ul></li>
<li class="chapter" data-level="9.4" data-path="9-hypothesis-testing.html"><a href="9-hypothesis-testing.html#ht-interpretation"><i class="fa fa-check"></i><b>9.4</b> Interpreting hypothesis tests</a>
<ul>
<li class="chapter" data-level="9.4.1" data-path="9-hypothesis-testing.html"><a href="9-hypothesis-testing.html#trial"><i class="fa fa-check"></i><b>9.4.1</b> Two possible outcomes</a></li>
<li class="chapter" data-level="9.4.2" data-path="9-hypothesis-testing.html"><a href="9-hypothesis-testing.html#types-of-errors"><i class="fa fa-check"></i><b>9.4.2</b> Types of errors</a></li>
<li class="chapter" data-level="9.4.3" data-path="9-hypothesis-testing.html"><a href="9-hypothesis-testing.html#choosing-alpha"><i class="fa fa-check"></i><b>9.4.3</b> How do we choose alpha?</a></li>
</ul></li>
<li class="chapter" data-level="9.5" data-path="9-hypothesis-testing.html"><a href="9-hypothesis-testing.html#ht-case-study"><i class="fa fa-check"></i><b>9.5</b> Case study: Are action or romance movies rated higher?</a>
<ul>
<li class="chapter" data-level="9.5.1" data-path="9-hypothesis-testing.html"><a href="9-hypothesis-testing.html#imdb-data"><i class="fa fa-check"></i><b>9.5.1</b> IMDb ratings data</a></li>
<li class="chapter" data-level="9.5.2" data-path="9-hypothesis-testing.html"><a href="9-hypothesis-testing.html#sampling-scenario-1"><i class="fa fa-check"></i><b>9.5.2</b> Sampling scenario</a></li>
<li class="chapter" data-level="9.5.3" data-path="9-hypothesis-testing.html"><a href="9-hypothesis-testing.html#conducting-the-hypothesis-test"><i class="fa fa-check"></i><b>9.5.3</b> Conducting the hypothesis test</a></li>
</ul></li>
<li class="chapter" data-level="9.6" data-path="9-hypothesis-testing.html"><a href="9-hypothesis-testing.html#nhst-conclusion"><i class="fa fa-check"></i><b>9.6</b> Conclusion</a>
<ul>
<li class="chapter" data-level="9.6.1" data-path="9-hypothesis-testing.html"><a href="9-hypothesis-testing.html#theory-hypo"><i class="fa fa-check"></i><b>9.6.1</b> Theory-based hypothesis tests</a></li>
<li class="chapter" data-level="9.6.2" data-path="9-hypothesis-testing.html"><a href="9-hypothesis-testing.html#when-inference-is-not-needed"><i class="fa fa-check"></i><b>9.6.2</b> When inference is not needed</a></li>
<li class="chapter" data-level="9.6.3" data-path="9-hypothesis-testing.html"><a href="9-hypothesis-testing.html#problems-with-p-values"><i class="fa fa-check"></i><b>9.6.3</b> Problems with p-values</a></li>
<li class="chapter" data-level="9.6.4" data-path="9-hypothesis-testing.html"><a href="9-hypothesis-testing.html#additional-resources-7"><i class="fa fa-check"></i><b>9.6.4</b> Additional resources</a></li>
<li class="chapter" data-level="9.6.5" data-path="9-hypothesis-testing.html"><a href="9-hypothesis-testing.html#whats-to-come-8"><i class="fa fa-check"></i><b>9.6.5</b> What’s to come</a></li>
</ul></li>
</ul></li>
<li class="chapter" data-level="10" data-path="10-inference-for-regression.html"><a href="10-inference-for-regression.html"><i class="fa fa-check"></i><b>10</b> Inference for Regression</a>
<ul>
<li class="chapter" data-level="" data-path="10-inference-for-regression.html"><a href="10-inference-for-regression.html#inf-packages"><i class="fa fa-check"></i>Needed packages</a></li>
<li class="chapter" data-level="10.1" data-path="10-inference-for-regression.html"><a href="10-inference-for-regression.html#regression-refresher"><i class="fa fa-check"></i><b>10.1</b> Regression refresher</a>
<ul>
<li class="chapter" data-level="10.1.1" data-path="10-inference-for-regression.html"><a href="10-inference-for-regression.html#teaching-evaluations-analysis"><i class="fa fa-check"></i><b>10.1.1</b> Teaching evaluations analysis</a></li>
<li class="chapter" data-level="10.1.2" data-path="10-inference-for-regression.html"><a href="10-inference-for-regression.html#sampling-scenario-2"><i class="fa fa-check"></i><b>10.1.2</b> Sampling scenario</a></li>
</ul></li>
<li class="chapter" data-level="10.2" data-path="10-inference-for-regression.html"><a href="10-inference-for-regression.html#regression-interp"><i class="fa fa-check"></i><b>10.2</b> Interpreting regression tables</a>
<ul>
<li class="chapter" data-level="10.2.1" data-path="10-inference-for-regression.html"><a href="10-inference-for-regression.html#regression-se"><i class="fa fa-check"></i><b>10.2.1</b> Standard error</a></li>
<li class="chapter" data-level="10.2.2" data-path="10-inference-for-regression.html"><a href="10-inference-for-regression.html#regression-test-statistic"><i class="fa fa-check"></i><b>10.2.2</b> Test statistic</a></li>
<li class="chapter" data-level="10.2.3" data-path="10-inference-for-regression.html"><a href="10-inference-for-regression.html#p-value"><i class="fa fa-check"></i><b>10.2.3</b> p-value</a></li>
<li class="chapter" data-level="10.2.4" data-path="10-inference-for-regression.html"><a href="10-inference-for-regression.html#confidence-interval"><i class="fa fa-check"></i><b>10.2.4</b> Confidence interval</a></li>
<li class="chapter" data-level="10.2.5" data-path="10-inference-for-regression.html"><a href="10-inference-for-regression.html#regression-table-computation"><i class="fa fa-check"></i><b>10.2.5</b> How does R compute the table?</a></li>
</ul></li>
<li class="chapter" data-level="10.3" data-path="10-inference-for-regression.html"><a href="10-inference-for-regression.html#regression-conditions"><i class="fa fa-check"></i><b>10.3</b> Conditions for inference for regression</a>
<ul>
<li class="chapter" data-level="10.3.1" data-path="10-inference-for-regression.html"><a href="10-inference-for-regression.html#residuals-refresher"><i class="fa fa-check"></i><b>10.3.1</b> Residuals refresher</a></li>
<li class="chapter" data-level="10.3.2" data-path="10-inference-for-regression.html"><a href="10-inference-for-regression.html#linearity-of-relationship"><i class="fa fa-check"></i><b>10.3.2</b> Linearity of relationship</a></li>
<li class="chapter" data-level="10.3.3" data-path="10-inference-for-regression.html"><a href="10-inference-for-regression.html#independence-of-residuals"><i class="fa fa-check"></i><b>10.3.3</b> Independence of residuals</a></li>
<li class="chapter" data-level="10.3.4" data-path="10-inference-for-regression.html"><a href="10-inference-for-regression.html#normality-of-residuals"><i class="fa fa-check"></i><b>10.3.4</b> Normality of residuals</a></li>
<li class="chapter" data-level="10.3.5" data-path="10-inference-for-regression.html"><a href="10-inference-for-regression.html#equality-of-variance"><i class="fa fa-check"></i><b>10.3.5</b> Equality of variance</a></li>
<li class="chapter" data-level="10.3.6" data-path="10-inference-for-regression.html"><a href="10-inference-for-regression.html#what-is-the-conclusion"><i class="fa fa-check"></i><b>10.3.6</b> What’s the conclusion?</a></li>
</ul></li>
<li class="chapter" data-level="10.4" data-path="10-inference-for-regression.html"><a href="10-inference-for-regression.html#infer-regression"><i class="fa fa-check"></i><b>10.4</b> Simulation-based inference for regression</a>
<ul>
<li class="chapter" data-level="10.4.1" data-path="10-inference-for-regression.html"><a href="10-inference-for-regression.html#confidence-interval-for-slope"><i class="fa fa-check"></i><b>10.4.1</b> Confidence interval for slope</a></li>
<li class="chapter" data-level="10.4.2" data-path="10-inference-for-regression.html"><a href="10-inference-for-regression.html#hypothesis-test-for-slope"><i class="fa fa-check"></i><b>10.4.2</b> Hypothesis test for slope</a></li>
</ul></li>
<li class="chapter" data-level="10.5" data-path="10-inference-for-regression.html"><a href="10-inference-for-regression.html#inference-conclusion"><i class="fa fa-check"></i><b>10.5</b> Conclusion</a>
<ul>
<li class="chapter" data-level="10.5.1" data-path="10-inference-for-regression.html"><a href="10-inference-for-regression.html#theory-regression"><i class="fa fa-check"></i><b>10.5.1</b> Theory-based inference for regression</a></li>
<li class="chapter" data-level="10.5.2" data-path="10-inference-for-regression.html"><a href="10-inference-for-regression.html#summary-of-statistical-inference"><i class="fa fa-check"></i><b>10.5.2</b> Summary of statistical inference</a></li>
<li class="chapter" data-level="10.5.3" data-path="10-inference-for-regression.html"><a href="10-inference-for-regression.html#additional-resources-8"><i class="fa fa-check"></i><b>10.5.3</b> Additional resources</a></li>
<li class="chapter" data-level="10.5.4" data-path="10-inference-for-regression.html"><a href="10-inference-for-regression.html#whats-to-come-9"><i class="fa fa-check"></i><b>10.5.4</b> What’s to come</a></li>
</ul></li>
</ul></li>
<li class="part"><span><b>IV Conclusion</b></span></li>
<li class="chapter" data-level="11" data-path="11-thinking-with-data.html"><a href="11-thinking-with-data.html"><i class="fa fa-check"></i><b>11</b> Tell Your Story with Data</a>
<ul>
<li class="chapter" data-level="11.1" data-path="11-thinking-with-data.html"><a href="11-thinking-with-data.html#review"><i class="fa fa-check"></i><b>11.1</b> Review</a>
<ul>
<li class="chapter" data-level="" data-path="11-thinking-with-data.html"><a href="11-thinking-with-data.html#story-packages"><i class="fa fa-check"></i>Needed packages</a></li>
</ul></li>
<li class="chapter" data-level="11.2" data-path="11-thinking-with-data.html"><a href="11-thinking-with-data.html#seattle-house-prices"><i class="fa fa-check"></i><b>11.2</b> Case study: Seattle house prices</a>
<ul>
<li class="chapter" data-level="11.2.1" data-path="11-thinking-with-data.html"><a href="11-thinking-with-data.html#house-prices-EDA-I"><i class="fa fa-check"></i><b>11.2.1</b> Exploratory data analysis: Part I</a></li>
<li class="chapter" data-level="11.2.2" data-path="11-thinking-with-data.html"><a href="11-thinking-with-data.html#house-prices-EDA-II"><i class="fa fa-check"></i><b>11.2.2</b> Exploratory data analysis: Part II</a></li>
<li class="chapter" data-level="11.2.3" data-path="11-thinking-with-data.html"><a href="11-thinking-with-data.html#house-prices-regression"><i class="fa fa-check"></i><b>11.2.3</b> Regression modeling</a></li>
<li class="chapter" data-level="11.2.4" data-path="11-thinking-with-data.html"><a href="11-thinking-with-data.html#house-prices-making-predictions"><i class="fa fa-check"></i><b>11.2.4</b> Making predictions</a></li>
</ul></li>
<li class="chapter" data-level="11.3" data-path="11-thinking-with-data.html"><a href="11-thinking-with-data.html#data-journalism"><i class="fa fa-check"></i><b>11.3</b> Case study: Effective data storytelling</a>
<ul>
<li class="chapter" data-level="11.3.1" data-path="11-thinking-with-data.html"><a href="11-thinking-with-data.html#bechdel-test-for-hollywood-gender-representation"><i class="fa fa-check"></i><b>11.3.1</b> Bechdel test for Hollywood gender representation</a></li>
<li class="chapter" data-level="11.3.2" data-path="11-thinking-with-data.html"><a href="11-thinking-with-data.html#us-births-in-1999"><i class="fa fa-check"></i><b>11.3.2</b> US Births in 1999</a></li>
<li class="chapter" data-level="11.3.3" data-path="11-thinking-with-data.html"><a href="11-thinking-with-data.html#scripts-of-r-code"><i class="fa fa-check"></i><b>11.3.3</b> Scripts of R code</a></li>
</ul></li>
<li class="chapter" data-level="" data-path="11-thinking-with-data.html"><a href="11-thinking-with-data.html#concluding-remarks"><i class="fa fa-check"></i>Concluding remarks</a></li>
</ul></li>
<li class="appendix"><span><b>Appendix</b></span></li>
<li class="chapter" data-level="A" data-path="A-appendixA.html"><a href="A-appendixA.html"><i class="fa fa-check"></i><b>A</b> Statistical Background</a>
<ul>
<li class="chapter" data-level="A.1" data-path="A-appendixA.html"><a href="A-appendixA.html#appendix-stat-terms"><i class="fa fa-check"></i><b>A.1</b> Basic statistical terms</a>
<ul>
<li class="chapter" data-level="A.1.1" data-path="A-appendixA.html"><a href="A-appendixA.html#mean"><i class="fa fa-check"></i><b>A.1.1</b> Mean</a></li>
<li class="chapter" data-level="A.1.2" data-path="A-appendixA.html"><a href="A-appendixA.html#median"><i class="fa fa-check"></i><b>A.1.2</b> Median</a></li>
<li class="chapter" data-level="A.1.3" data-path="A-appendixA.html"><a href="A-appendixA.html#appendix-sd-variance"><i class="fa fa-check"></i><b>A.1.3</b> Standard deviation and variance</a></li>
<li class="chapter" data-level="A.1.4" data-path="A-appendixA.html"><a href="A-appendixA.html#five-number-summary"><i class="fa fa-check"></i><b>A.1.4</b> Five-number summary</a></li>
<li class="chapter" data-level="A.1.5" data-path="A-appendixA.html"><a href="A-appendixA.html#distribution"><i class="fa fa-check"></i><b>A.1.5</b> Distribution</a></li>
<li class="chapter" data-level="A.1.6" data-path="A-appendixA.html"><a href="A-appendixA.html#outliers"><i class="fa fa-check"></i><b>A.1.6</b> Outliers</a></li>
</ul></li>
<li class="chapter" data-level="A.2" data-path="A-appendixA.html"><a href="A-appendixA.html#appendix-normal-curve"><i class="fa fa-check"></i><b>A.2</b> Normal distribution</a></li>
<li class="chapter" data-level="A.3" data-path="A-appendixA.html"><a href="A-appendixA.html#appendix-log10-transformations"><i class="fa fa-check"></i><b>A.3</b> log10 transformations</a></li>
</ul></li>
<li class="chapter" data-level="B" data-path="B-appendixB.html"><a href="B-appendixB.html"><i class="fa fa-check"></i><b>B</b> Inference Examples</a>
<ul>
<li class="chapter" data-level="" data-path="B-appendixB.html"><a href="B-appendixB.html#needed-packages-1"><i class="fa fa-check"></i>Needed packages</a></li>
<li class="chapter" data-level="B.1" data-path="B-appendixB.html"><a href="B-appendixB.html#inference-mind-map"><i class="fa fa-check"></i><b>B.1</b> Inference mind map</a></li>
<li class="chapter" data-level="B.2" data-path="B-appendixB.html"><a href="B-appendixB.html#one-mean"><i class="fa fa-check"></i><b>B.2</b> One mean</a>
<ul>
<li class="chapter" data-level="B.2.1" data-path="B-appendixB.html"><a href="B-appendixB.html#problem-statement"><i class="fa fa-check"></i><b>B.2.1</b> Problem statement</a></li>
<li class="chapter" data-level="B.2.2" data-path="B-appendixB.html"><a href="B-appendixB.html#competing-hypotheses"><i class="fa fa-check"></i><b>B.2.2</b> Competing hypotheses</a></li>
<li class="chapter" data-level="B.2.3" data-path="B-appendixB.html"><a href="B-appendixB.html#exploring-the-sample-data"><i class="fa fa-check"></i><b>B.2.3</b> Exploring the sample data</a></li>
<li class="chapter" data-level="B.2.4" data-path="B-appendixB.html"><a href="B-appendixB.html#non-traditional-methods"><i class="fa fa-check"></i><b>B.2.4</b> Non-traditional methods</a></li>
<li class="chapter" data-level="B.2.5" data-path="B-appendixB.html"><a href="B-appendixB.html#traditional-methods"><i class="fa fa-check"></i><b>B.2.5</b> Traditional methods</a></li>
<li class="chapter" data-level="B.2.6" data-path="B-appendixB.html"><a href="B-appendixB.html#comparing-results"><i class="fa fa-check"></i><b>B.2.6</b> Comparing results</a></li>
</ul></li>
<li class="chapter" data-level="B.3" data-path="B-appendixB.html"><a href="B-appendixB.html#one-proportion"><i class="fa fa-check"></i><b>B.3</b> One proportion</a>
<ul>
<li class="chapter" data-level="B.3.1" data-path="B-appendixB.html"><a href="B-appendixB.html#problem-statement-1"><i class="fa fa-check"></i><b>B.3.1</b> Problem statement</a></li>
<li class="chapter" data-level="B.3.2" data-path="B-appendixB.html"><a href="B-appendixB.html#competing-hypotheses-1"><i class="fa fa-check"></i><b>B.3.2</b> Competing hypotheses</a></li>
<li class="chapter" data-level="B.3.3" data-path="B-appendixB.html"><a href="B-appendixB.html#exploring-the-sample-data-1"><i class="fa fa-check"></i><b>B.3.3</b> Exploring the sample data</a></li>
<li class="chapter" data-level="B.3.4" data-path="B-appendixB.html"><a href="B-appendixB.html#non-traditional-methods-1"><i class="fa fa-check"></i><b>B.3.4</b> Non-traditional methods</a></li>
<li class="chapter" data-level="B.3.5" data-path="B-appendixB.html"><a href="B-appendixB.html#traditional-methods-1"><i class="fa fa-check"></i><b>B.3.5</b> Traditional methods</a></li>
<li class="chapter" data-level="B.3.6" data-path="B-appendixB.html"><a href="B-appendixB.html#comparing-results-1"><i class="fa fa-check"></i><b>B.3.6</b> Comparing results</a></li>
</ul></li>
<li class="chapter" data-level="B.4" data-path="B-appendixB.html"><a href="B-appendixB.html#two-proportions"><i class="fa fa-check"></i><b>B.4</b> Two proportions</a>
<ul>
<li class="chapter" data-level="B.4.1" data-path="B-appendixB.html"><a href="B-appendixB.html#problem-statement-2"><i class="fa fa-check"></i><b>B.4.1</b> Problem statement</a></li>
<li class="chapter" data-level="B.4.2" data-path="B-appendixB.html"><a href="B-appendixB.html#competing-hypotheses-2"><i class="fa fa-check"></i><b>B.4.2</b> Competing hypotheses</a></li>
<li class="chapter" data-level="B.4.3" data-path="B-appendixB.html"><a href="B-appendixB.html#exploring-the-sample-data-2"><i class="fa fa-check"></i><b>B.4.3</b> Exploring the sample data</a></li>
<li class="chapter" data-level="B.4.4" data-path="B-appendixB.html"><a href="B-appendixB.html#non-traditional-methods-2"><i class="fa fa-check"></i><b>B.4.4</b> Non-traditional methods</a></li>
<li class="chapter" data-level="B.4.5" data-path="B-appendixB.html"><a href="B-appendixB.html#traditional-methods-2"><i class="fa fa-check"></i><b>B.4.5</b> Traditional methods</a></li>
<li class="chapter" data-level="B.4.6" data-path="B-appendixB.html"><a href="B-appendixB.html#test-statistic-2"><i class="fa fa-check"></i><b>B.4.6</b> Test statistic</a></li>
<li class="chapter" data-level="B.4.7" data-path="B-appendixB.html"><a href="B-appendixB.html#state-conclusion-2"><i class="fa fa-check"></i><b>B.4.7</b> State conclusion</a></li>
<li class="chapter" data-level="B.4.8" data-path="B-appendixB.html"><a href="B-appendixB.html#comparing-results-2"><i class="fa fa-check"></i><b>B.4.8</b> Comparing results</a></li>
</ul></li>
<li class="chapter" data-level="B.5" data-path="B-appendixB.html"><a href="B-appendixB.html#two-means-independent-samples"><i class="fa fa-check"></i><b>B.5</b> Two means (independent samples)</a>
<ul>
<li class="chapter" data-level="B.5.1" data-path="B-appendixB.html"><a href="B-appendixB.html#problem-statement-3"><i class="fa fa-check"></i><b>B.5.1</b> Problem statement</a></li>
<li class="chapter" data-level="B.5.2" data-path="B-appendixB.html"><a href="B-appendixB.html#competing-hypotheses-3"><i class="fa fa-check"></i><b>B.5.2</b> Competing hypotheses</a></li>
<li class="chapter" data-level="B.5.3" data-path="B-appendixB.html"><a href="B-appendixB.html#exploring-the-sample-data-3"><i class="fa fa-check"></i><b>B.5.3</b> Exploring the sample data</a></li>
<li class="chapter" data-level="B.5.4" data-path="B-appendixB.html"><a href="B-appendixB.html#non-traditional-methods-3"><i class="fa fa-check"></i><b>B.5.4</b> Non-traditional methods</a></li>
<li class="chapter" data-level="B.5.5" data-path="B-appendixB.html"><a href="B-appendixB.html#traditional-methods-3"><i class="fa fa-check"></i><b>B.5.5</b> Traditional methods</a></li>
<li class="chapter" data-level="B.5.6" data-path="B-appendixB.html"><a href="B-appendixB.html#test-statistic-3"><i class="fa fa-check"></i><b>B.5.6</b> Test statistic</a></li>
<li class="chapter" data-level="B.5.7" data-path="B-appendixB.html"><a href="B-appendixB.html#compute-p-value-1"><i class="fa fa-check"></i><b>B.5.7</b> Compute <span class="math inline">\(p\)</span>-value</a></li>
<li class="chapter" data-level="B.5.8" data-path="B-appendixB.html"><a href="B-appendixB.html#state-conclusion-3"><i class="fa fa-check"></i><b>B.5.8</b> State conclusion</a></li>
<li class="chapter" data-level="B.5.9" data-path="B-appendixB.html"><a href="B-appendixB.html#comparing-results-3"><i class="fa fa-check"></i><b>B.5.9</b> Comparing results</a></li>
</ul></li>
<li class="chapter" data-level="B.6" data-path="B-appendixB.html"><a href="B-appendixB.html#two-means-paired-samples"><i class="fa fa-check"></i><b>B.6</b> Two means (paired samples)</a>
<ul>
<li class="chapter" data-level="" data-path="B-appendixB.html"><a href="B-appendixB.html#problem-statement-4"><i class="fa fa-check"></i>Problem statement</a></li>
<li class="chapter" data-level="B.6.1" data-path="B-appendixB.html"><a href="B-appendixB.html#competing-hypotheses-4"><i class="fa fa-check"></i><b>B.6.1</b> Competing hypotheses</a></li>
<li class="chapter" data-level="B.6.2" data-path="B-appendixB.html"><a href="B-appendixB.html#exploring-the-sample-data-4"><i class="fa fa-check"></i><b>B.6.2</b> Exploring the sample data</a></li>
<li class="chapter" data-level="B.6.3" data-path="B-appendixB.html"><a href="B-appendixB.html#non-traditional-methods-4"><i class="fa fa-check"></i><b>B.6.3</b> Non-traditional methods</a></li>
<li class="chapter" data-level="B.6.4" data-path="B-appendixB.html"><a href="B-appendixB.html#traditional-methods-4"><i class="fa fa-check"></i><b>B.6.4</b> Traditional methods</a></li>
<li class="chapter" data-level="B.6.5" data-path="B-appendixB.html"><a href="B-appendixB.html#comparing-results-4"><i class="fa fa-check"></i><b>B.6.5</b> Comparing results</a></li>
</ul></li>
</ul></li>
<li class="chapter" data-level="C" data-path="C-appendixC.html"><a href="C-appendixC.html"><i class="fa fa-check"></i><b>C</b> Tips and Tricks</a>
<ul>
<li class="chapter" data-level="" data-path="C-appendixC.html"><a href="C-appendixC.html#needed-packages-2"><i class="fa fa-check"></i>Needed packages</a></li>
<li class="chapter" data-level="C.1" data-path="C-appendixC.html"><a href="C-appendixC.html#data-wrangling"><i class="fa fa-check"></i><b>C.1</b> Data wrangling</a>
<ul>
<li class="chapter" data-level="C.1.1" data-path="C-appendixC.html"><a href="C-appendixC.html#appendix-missing-values"><i class="fa fa-check"></i><b>C.1.1</b> Dealing with missing values</a></li>
<li class="chapter" data-level="C.1.2" data-path="C-appendixC.html"><a href="C-appendixC.html#appendix-reordering-bars"><i class="fa fa-check"></i><b>C.1.2</b> Reordering bars in a barplot</a></li>
<li class="chapter" data-level="C.1.3" data-path="C-appendixC.html"><a href="C-appendixC.html#appendix-money-on-axis"><i class="fa fa-check"></i><b>C.1.3</b> Showing money on an axis</a></li>
<li class="chapter" data-level="C.1.4" data-path="C-appendixC.html"><a href="C-appendixC.html#appendix-changing-values"><i class="fa fa-check"></i><b>C.1.4</b> Changing values inside cells</a></li>
<li class="chapter" data-level="C.1.5" data-path="C-appendixC.html"><a href="C-appendixC.html#appendix-convert-numerical-categorical"><i class="fa fa-check"></i><b>C.1.5</b> Converting a numerical variable to a categorical one</a></li>
<li class="chapter" data-level="C.1.6" data-path="C-appendixC.html"><a href="C-appendixC.html#appendix-prop"><i class="fa fa-check"></i><b>C.1.6</b> Computing proportions</a></li>
<li class="chapter" data-level="C.1.7" data-path="C-appendixC.html"><a href="C-appendixC.html#appendix-commas"><i class="fa fa-check"></i><b>C.1.7</b> Dealing with %, commas, and $</a></li>
</ul></li>
<li class="chapter" data-level="C.2" data-path="C-appendixC.html"><a href="C-appendixC.html#interactive-graphics"><i class="fa fa-check"></i><b>C.2</b> Interactive graphics</a>
<ul>
<li class="chapter" data-level="C.2.1" data-path="C-appendixC.html"><a href="C-appendixC.html#interactive-linegraphs"><i class="fa fa-check"></i><b>C.2.1</b> Interactive linegraphs</a></li>
</ul></li>
</ul></li>
<li class="chapter" data-level="D" data-path="D-appendixD.html"><a href="D-appendixD.html"><i class="fa fa-check"></i><b>D</b> Learning Check Solutions</a>
<ul>
<li class="chapter" data-level="D.1" data-path="D-appendixD.html"><a href="D-appendixD.html#chapter-1-solutions"><i class="fa fa-check"></i><b>D.1</b> Chapter 1 Solutions</a></li>
</ul></li>
<li class="chapter" data-level="E" data-path="E-appendixE.html"><a href="E-appendixE.html"><i class="fa fa-check"></i><b>E</b> Versions of R Packages Used</a></li>
<li class="chapter" data-level="" data-path="references.html"><a href="references.html"><i class="fa fa-check"></i>References</a></li>
</ul>
</nav>
</div>
<div class="book-body">
<div class="body-inner">
<div class="book-header" role="navigation">
<h1>
<i class="fa fa-circle-o-notch fa-spin"></i><a href="./">Statistical Inference via Data Science</a>
</h1>
</div>
<div class="page-wrapper" tabindex="-1" role="main">
<div class="page-inner">
<section class="normal" id="section-">
<html>
<img src='https://moderndive.com/wide_format.png' alt="ModernDive">
</html>
<div id="wrangling" class="section level1" number="3">
<h1><span class="header-section-number">Chapter 3</span> Data Wrangling</h1>
<p>So far in our journey, we’ve seen how to look at data saved in data frames using the <code>glimpse()</code> and <code>View()</code> functions in Chapter <a href="1-getting-started.html#getting-started">1</a>, and how to create data visualizations using the <code>ggplot2</code> package in Chapter <a href="2-viz.html#viz">2</a>. In particular we studied what we term the “five named graphs” (5NG):</p>
<ol style="list-style-type: decimal">
<li>scatterplots via <code>geom_point()</code></li>
<li>linegraphs via <code>geom_line()</code></li>
<li>boxplots via <code>geom_boxplot()</code></li>
<li>histograms via <code>geom_histogram()</code></li>
<li>barplots via <code>geom_bar()</code> or <code>geom_col()</code></li>
</ol>
<p>We created these visualizations using the grammar of graphics, which maps variables in a data frame to the aesthetic attributes of one of the 5 <code>geom</code>etric objects. We can also control other aesthetic attributes of the geometric objects such as the size and color as seen in the Gapminder data example in Figure <a href="2-viz.html#fig:gapminder">2.1</a>.</p>
<p>Recall however that for two of our visualizations, we first needed to transform/modify existing data frames a little. For example, recall the scatterplot in Figure <a href="2-viz.html#fig:noalpha">2.2</a> of length and weight measurements <em>only</em> for brackish fish species. In order to create this visualization, we first needed to pare down the <code>all_fishdata</code> data frame to a <code>brackish_fish</code> data frame consisting of only <code>Brack == "1"</code> fish species Thus, <code>brackish_fish</code> will have fewer rows than <code>all_fishdata</code>. We did this using the <code>filter()</code> function:</p>
<div class="sourceCode" id="cb49"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb49-1"><a href="3-wrangling.html#cb49-1" aria-hidden="true" tabindex="-1"></a>brackish_fish <span class="ot"><-</span> all_fishdata <span class="sc">%>%</span> </span>
<span id="cb49-2"><a href="3-wrangling.html#cb49-2" aria-hidden="true" tabindex="-1"></a> <span class="fu">filter</span>(Brack <span class="sc">==</span> <span class="dv">1</span>)</span></code></pre></div>
<p>In this chapter, we’ll extend this example and we’ll introduce a series of functions from the <code>dplyr</code> package for data wrangling that will allow you to take a data frame and “wrangle” it (transform it) to suit your needs. Such functions include:</p>
<ol style="list-style-type: decimal">
<li><code>filter()</code> and <code>slice</code> a data frame’s existing rows to only pick out a subset of them. For example, the <code>brackish_fish</code> data frame.</li>
<li><code>select()</code> a data frame’s existing columns to only pick out a subset of them or <code>rename</code> existing columns.</li>
<li><code>summarize()</code> one or more of its columns/variables with a <em>summary statistic</em>. Examples of summary statistics include the median and interquartile range of chick weights as we saw in Section <a href="2-viz.html#boxplots">2.7</a> on boxplots.</li>
<li><code>group_by()</code> its rows. In other words, assign different rows to be part of the same <em>group</em>. We can then combine <code>group_by()</code> with <code>summarize()</code> to report summary statistics for each group <em>separately</em>. For example, say you don’t want a single overall average <code>weight</code> on Day 21 for the <code>chick_weight_d21</code> dataset, but rather four separate averages, one computed for each of the four <code>Diet</code> groups.</li>
<li><code>mutate()</code> its existing columns/variables to create new ones. For example, convert weight recordings from grams to ounces.</li>
<li><code>arrange()</code> its rows. For example, sort the rows of <code>all_fishdata</code> in ascending or descending order of <code>Length</code>.</li>
<li><code>join()</code> it with another data frame by matching along a “key” variable. In other words, merge these two data frames together.</li>
</ol>
<p>Notice how we used <code>computer_code</code> font to describe the actions we want to take on our data frames. This is because the <code>dplyr</code> package for data wrangling has intuitively verb-named functions that are easy to remember.</p>
<p>There is a further benefit to learning to use the <code>dplyr</code> package for data wrangling: its similarity to the database querying language <a href="https://en.wikipedia.org/wiki/SQL">SQL</a> (pronounced “sequel” or spelled out as “S,” “Q,” “L”). SQL (which stands for “Structured Query Language”) is used to manage large databases quickly and efficiently and is widely used by many institutions with a lot of data. While SQL is a topic left for a book or a course on database management, keep in mind that once you learn <code>dplyr</code>, you can learn SQL easily. We’ll talk more about their similarities in Subsection <a href="#normal-forms"><strong>??</strong></a>.</p>
<div id="wrangling-packages" class="section level3 unnumbered">
<h3>Needed packages</h3>
<p>Let’s load all the packages needed for this chapter (this assumes you’ve already installed them). If needed, read Section <a href="1-getting-started.html#packages">1.3</a> for information on how to install and load R packages.</p>
<div class="sourceCode" id="cb50"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb50-1"><a href="3-wrangling.html#cb50-1" aria-hidden="true" tabindex="-1"></a><span class="fu">library</span>(dplyr)</span>
<span id="cb50-2"><a href="3-wrangling.html#cb50-2" aria-hidden="true" tabindex="-1"></a><span class="fu">library</span>(ggplot2)</span>
<span id="cb50-3"><a href="3-wrangling.html#cb50-3" aria-hidden="true" tabindex="-1"></a><span class="fu">library</span>(rfishbase)</span></code></pre></div>
</div>
<div id="piping" class="section level2" number="3.1">
<h2><span class="header-section-number">3.1</span> The pipe operator: <code>%>%</code></h2>
<p>Before we start data wrangling, let’s first introduce a nifty tool that gets loaded with the <code>dplyr</code> package: the pipe operator <code>%>%</code>. The pipe operator allows us to combine multiple operations in R into a single sequential <em>chain</em> of actions.</p>
<p>Let’s start with a hypothetical example. Say you would like to perform a hypothetical sequence of operations on a hypothetical data frame <code>x</code> using hypothetical functions <code>f()</code>, <code>g()</code>, and <code>h()</code>:</p>
<ol style="list-style-type: decimal">
<li>Take <code>x</code> <em>then</em></li>
<li>Use <code>x</code> as an input to a function <code>f()</code> <em>then</em></li>
<li>Use the output of <code>f(x)</code> as an input to a function <code>g()</code> <em>then</em></li>
<li>Use the output of <code>g(f(x))</code> as an input to a function <code>h()</code></li>
</ol>
<p>One way to achieve this sequence of operations is by using nesting parentheses as follows:</p>
<div class="sourceCode" id="cb51"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb51-1"><a href="3-wrangling.html#cb51-1" aria-hidden="true" tabindex="-1"></a><span class="fu">h</span>(<span class="fu">g</span>(<span class="fu">f</span>(x)))</span></code></pre></div>
<p>This code isn’t so hard to read since we are applying only three functions: <code>f()</code>, then <code>g()</code>, then <code>h()</code> and each of the functions is short in its name. Further, each of these functions also only has one argument. However, you can imagine that this will get progressively harder to read as the number of functions applied in your sequence increases and the arguments in each function increase as well. This is where the pipe operator <code>%>%</code> comes in handy. <code>%>%</code> takes the output of one function and then “pipes” it to be the input of the next function. Furthermore, a helpful trick is to read <code>%>%</code> as “then” or “and then.” For example, you can obtain the same output as the hypothetical sequence of functions as follows:</p>
<div class="sourceCode" id="cb52"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb52-1"><a href="3-wrangling.html#cb52-1" aria-hidden="true" tabindex="-1"></a>x <span class="sc">%>%</span> </span>
<span id="cb52-2"><a href="3-wrangling.html#cb52-2" aria-hidden="true" tabindex="-1"></a> <span class="fu">f</span>() <span class="sc">%>%</span> </span>
<span id="cb52-3"><a href="3-wrangling.html#cb52-3" aria-hidden="true" tabindex="-1"></a> <span class="fu">g</span>() <span class="sc">%>%</span> </span>
<span id="cb52-4"><a href="3-wrangling.html#cb52-4" aria-hidden="true" tabindex="-1"></a> <span class="fu">h</span>()</span></code></pre></div>
<p>You would read this sequence as:</p>
<ol style="list-style-type: decimal">
<li>Take <code>x</code> <em>then</em></li>
<li>Use this output as the input to the next function <code>f()</code> <em>then</em></li>
<li>Use this output as the input to the next function <code>g()</code> <em>then</em></li>
<li>Use this output as the input to the next function <code>h()</code></li>
</ol>
<p>So while both approaches achieve the same goal, the latter is much more human-readable because you can clearly read the sequence of operations line-by-line. But what are the hypothetical <code>x</code>, <code>f()</code>, <code>g()</code>, and <code>h()</code>? Throughout this chapter on data wrangling:</p>
<ol style="list-style-type: decimal">
<li>The starting value <code>x</code> will be a data frame. For example, the <code>all_fishdata</code> data frame we explored in Section <a href="1-getting-started.html#fishbasedataframe">1.4.2</a>.</li>
<li>The sequence of functions, here <code>f()</code>, <code>g()</code>, and <code>h()</code>, will mostly be a sequence of any number of the data wrangling verb-named functions we listed in the introduction to this chapter. For example, the <code>filter(Brack == "1")</code> function and argument we previewed earlier.</li>
<li>The result will be the transformed/modified data frame that you want. In our example, we’ll save the result in a new data frame by using the <code><-</code> assignment operator with the name <code>brackish_fish</code> via <code>brackish_fish <-</code>.</li>
</ol>
<div class="sourceCode" id="cb53"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb53-1"><a href="3-wrangling.html#cb53-1" aria-hidden="true" tabindex="-1"></a>brackish_fish <span class="ot"><-</span> all_fishdata <span class="sc">%>%</span> </span>
<span id="cb53-2"><a href="3-wrangling.html#cb53-2" aria-hidden="true" tabindex="-1"></a> <span class="fu">filter</span>(Brack <span class="sc">==</span> <span class="st">"1"</span>)</span></code></pre></div>
<p>Much like when adding layers to a <code>ggplot()</code> using the <code>+</code> sign, you form a single <em>chain</em> of data wrangling operations by combining verb-named functions into a single sequence using the pipe operator <code>%>%</code>. Furthermore, much like how the <code>+</code> sign has to come at the end of lines when constructing plots, the pipe operator <code>%>%</code> has to come at the end of lines as well.</p>
<p>Keep in mind, there are many more advanced data wrangling functions than just those listed in the introduction to this chapter. However, just with these verb-named functions you’ll be able to perform a broad array of data wrangling tasks for the rest of this book.</p>
</div>
<div id="filter" class="section level2" number="3.2">
<h2><span class="header-section-number">3.2</span> <code>filter</code> rows</h2>
<div class="figure" style="text-align: center"><span id="fig:filter"></span>
<img src="images/cheatsheets/filter.png" alt="Diagram of filter() rows operation." width="\textwidth" />
<p class="caption">
FIGURE 3.1: Diagram of filter() rows operation.
</p>
</div>
<p>The <code>filter()</code> function here works much like the “Filter” option in Microsoft Excel; it allows you to specify criteria about the values of a variable in your dataset and then filters out only the rows that match that criteria.</p>
<p>We begin by focusing only on fish species found in <code>"public aquariums"</code>. Run the following and look at the results in RStudio’s spreadsheet viewer to ensure that only fish species in public aquariums are chosen here:</p>
<div class="sourceCode" id="cb54"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb54-1"><a href="3-wrangling.html#cb54-1" aria-hidden="true" tabindex="-1"></a>paquarium_fish <span class="ot"><-</span> all_fishdata <span class="sc">%>%</span> </span>
<span id="cb54-2"><a href="3-wrangling.html#cb54-2" aria-hidden="true" tabindex="-1"></a> <span class="fu">filter</span>(Aquarium <span class="sc">==</span> <span class="st">"public aquariums"</span>)</span>
<span id="cb54-3"><a href="3-wrangling.html#cb54-3" aria-hidden="true" tabindex="-1"></a><span class="fu">View</span>(paquarium_fish)</span></code></pre></div>
<p>Note the order of the code. First, take the <code>all_fishdata</code> data frame; <em>then</em> <code>filter()</code> the data frame so that only those where <code>Aquarium</code> equals <code>"public aquariums"</code> are included. We test for equality using the double equal sign <code>==</code> and not a single equal sign <code>=</code>. In other words <code>filter(Aquarium = "public aquariums")</code> will yield an error. This is a convention across many programming languages. If you are new to coding, you’ll probably forget to use the double equal sign <code>==</code> a few times before you get the hang of it.</p>
<p>You can use other operators beyond just the <code>==</code> operator that tests for equality:</p>
<ul>
<li><code>></code> corresponds to “greater than”</li>
<li><code><</code> corresponds to “less than”</li>
<li><code>>=</code> corresponds to “greater than or equal to”</li>
<li><code><=</code> corresponds to “less than or equal to”</li>
<li><code>!=</code> corresponds to “not equal to.” The <code>!</code> is used in many programming languages to indicate “not.”</li>
</ul>
<p>Furthermore, you can combine multiple criteria using operators that make comparisons:</p>
<ul>
<li><code>|</code> corresponds to “or”</li>
<li><code>&</code> corresponds to “and”</li>
</ul>
<p>To see many of these in action, let’s filter <code>all_fishdata</code> for aquarium fish species that are <code>"commercial"</code> or <code>"highly commercial"</code> <em>and</em> may be dangerous (not <code>"harmless"</code> and not<code>"Harmless"</code>). Note that this example uses the <code>!</code> “not” operator to pick rows that <em>don’t</em> match a criteria. As mentioned earlier, the <code>!</code> can be read as “not.” Run the following:</p>
<div class="sourceCode" id="cb55"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb55-1"><a href="3-wrangling.html#cb55-1" aria-hidden="true" tabindex="-1"></a>danger_comm_fish <span class="ot"><-</span> all_fishdata <span class="sc">%>%</span> </span>
<span id="cb55-2"><a href="3-wrangling.html#cb55-2" aria-hidden="true" tabindex="-1"></a> <span class="fu">filter</span>((Aquarium <span class="sc">==</span> <span class="st">"commercial"</span> <span class="sc">|</span> Aquarium <span class="sc">==</span> <span class="st">"highly commercial"</span>) <span class="sc">&</span> Dangerous <span class="sc">!=</span> <span class="st">"harmless"</span> <span class="sc">&</span> Dangerous <span class="sc">!=</span> <span class="st">"Harmless"</span>)</span>
<span id="cb55-3"><a href="3-wrangling.html#cb55-3" aria-hidden="true" tabindex="-1"></a><span class="fu">View</span>(danger_comm_fish)</span></code></pre></div>
<p>This alternative code where we do <em>not</em> select fish species that are <code>"harmless"</code> or <code>"Harmless"</code> achieves the same aim:</p>
<div class="sourceCode" id="cb56"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb56-1"><a href="3-wrangling.html#cb56-1" aria-hidden="true" tabindex="-1"></a>danger_comm_fish <span class="ot"><-</span> all_fishdata <span class="sc">%>%</span> </span>
<span id="cb56-2"><a href="3-wrangling.html#cb56-2" aria-hidden="true" tabindex="-1"></a> <span class="fu">filter</span>((Aquarium <span class="sc">==</span> <span class="st">"commercial"</span> <span class="sc">|</span> Aquarium <span class="sc">==</span> <span class="st">"highly commercial"</span>) <span class="sc">&</span> <span class="sc">!</span>(Dangerous <span class="sc">==</span> <span class="st">"harmless"</span> <span class="sc">|</span> Dangerous <span class="sc">==</span> <span class="st">"Harmless"</span>))</span>
<span id="cb56-3"><a href="3-wrangling.html#cb56-3" aria-hidden="true" tabindex="-1"></a><span class="fu">View</span>(danger_comm_fish)</span></code></pre></div>
<p>Note that even though colloquially speaking one might say “all fish species that are commercial <em>and</em> highly commercial,” in terms of computer operations, we really mean “all fish species that are commercial <em>or</em> highly commercial.” For a given row in the data, <code>Aquarium</code> can be <code>"commercial"</code>, or <code>"highly commercial"</code>, or something else, but not both <code>"commercial"</code> and <code>"highly commercial"</code> at the same time. Furthermore, note the careful use of parentheses around <code>Aquarium == "commercial" | Aquarium == "highly commercial"</code> and <code>Dangerous == "harmless" | Dangerous == "Harmless"</code>.</p>
<p>We can often skip the use of <code>&</code> and just separate our conditions with a comma. The following code will return the identical output <code>danger_comm_fish</code> as the previous code:</p>
<div class="sourceCode" id="cb57"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb57-1"><a href="3-wrangling.html#cb57-1" aria-hidden="true" tabindex="-1"></a>danger_comm_fish <span class="ot"><-</span> all_fishdata <span class="sc">%>%</span> </span>
<span id="cb57-2"><a href="3-wrangling.html#cb57-2" aria-hidden="true" tabindex="-1"></a> <span class="fu">filter</span>((Aquarium <span class="sc">==</span> <span class="st">"commercial"</span> <span class="sc">|</span> Aquarium <span class="sc">==</span> <span class="st">"highly commercial"</span>), <span class="sc">!</span>(Dangerous <span class="sc">==</span> <span class="st">"harmless"</span> <span class="sc">|</span> Dangerous <span class="sc">==</span> <span class="st">"Harmless"</span>))</span>
<span id="cb57-3"><a href="3-wrangling.html#cb57-3" aria-hidden="true" tabindex="-1"></a><span class="fu">View</span>(danger_comm_fish)</span></code></pre></div>
<p>Now say we have a larger number of categories we want to filter for, say <code>"commercial"</code>, <code>"highly commercial"</code>, <code>"show aquarium"</code>, and <code>"public aquariums"</code>. We could continue to use the <code>|</code> (<em>or</em>) operator:</p>
<div class="sourceCode" id="cb58"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb58-1"><a href="3-wrangling.html#cb58-1" aria-hidden="true" tabindex="-1"></a>many_aquarium_fish <span class="ot"><-</span> all_fishdata <span class="sc">%>%</span> </span>
<span id="cb58-2"><a href="3-wrangling.html#cb58-2" aria-hidden="true" tabindex="-1"></a> <span class="fu">filter</span>(Aquarium <span class="sc">==</span> <span class="st">"commercial"</span> <span class="sc">|</span> Aquarium <span class="sc">==</span> <span class="st">"highly commercial"</span> <span class="sc">|</span> Aquarium <span class="sc">==</span> <span class="st">"show aquarium"</span> <span class="sc">|</span> Aquarium <span class="sc">==</span> <span class="st">"public aquariums"</span>)</span></code></pre></div>
<p>but as we progressively include more categories, this will get unwieldy to write. A slightly shorter approach uses the <code>%in%</code> operator along with the <code>c()</code> function. Recall from Subsection <a href="1-getting-started.html#programming-concepts">1.2.1</a> that the <code>c()</code> function “combines” or “concatenates” values into a single <em>vector</em> of values. </p>
<div class="sourceCode" id="cb59"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb59-1"><a href="3-wrangling.html#cb59-1" aria-hidden="true" tabindex="-1"></a>many_aquarium_fish <span class="ot"><-</span> all_fishdata <span class="sc">%>%</span> </span>
<span id="cb59-2"><a href="3-wrangling.html#cb59-2" aria-hidden="true" tabindex="-1"></a> <span class="fu">filter</span>(Aquarium <span class="sc">%in%</span> <span class="fu">c</span>(<span class="st">"commercial"</span>, <span class="st">"highly commercial"</span>, <span class="st">"show aquarium"</span>, <span class="st">"public aquariums"</span>))</span>
<span id="cb59-3"><a href="3-wrangling.html#cb59-3" aria-hidden="true" tabindex="-1"></a><span class="fu">View</span>(many_aquarium_fish)</span></code></pre></div>
<p>This code filters <code>all_fishdata</code> for all fish species where <code>Aquarium</code> is in the vector of types <code>c("commercial", "highly commercial", "show aquarium", "public aquariums")</code>. Both outputs of <code>many_aquarium_fish</code> are the same, but as you can see the latter takes much less energy to code. The <code>%in%</code> operator is useful for looking for matches commonly in one vector/variable compared to another.</p>
<p>As a final note, we recommend that <code>filter()</code> should often be among the first verbs you consider applying to your data. This cleans your dataset to only those rows you care about, or put differently, it narrows down the scope of your data frame to just the observations you care about.</p>
<div class="learncheck">
<p>
<strong><em>Learning check</em></strong>
</p>
</div>
<p><strong>(LC3.1)</strong> Adapt the previous code using the “not” operator <code>!</code> to filter only the fish species that are not <code>commercial</code> or <code>highly commercial</code> in the <code>all_fishdata</code> data frame.</p>
<div class="learncheck">
</div>
</div>
<div id="slice-rows" class="section level2" number="3.3">
<h2><span class="header-section-number">3.3</span> <code>slice</code> rows</h2>
<p>Similar to <code>filter</code>, the <code>slice</code> function function returns a subset of row from a data frame. While <code>filter</code> returns the rows that match a specified criteria about the values of a variable (e.g., <code>Aquarium == "public aquariums"</code>), the <code>slice</code> function returns rows based on their positions. For example, let’s <code>slice</code> the first 100 rows of the <code>all_fishdata</code> data frame:</p>
<div class="sourceCode" id="cb60"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb60-1"><a href="3-wrangling.html#cb60-1" aria-hidden="true" tabindex="-1"></a>all_fishdata <span class="sc">%>%</span> <span class="fu">slice</span>(<span class="dv">1</span><span class="sc">:</span><span class="dv">100</span>)</span></code></pre></div>
<p>However, even more useful is a derivative of <code>slice</code> called <code>slice_max</code> that allow us to retrieve rows with the top values of a specified variable. For example, we can return a data frame of the 10 fish species found at the deepest depths. Observe that we set the number of values to return to <code>n = 10</code> and <code>order_by = DepthRangeDeep</code> to indicate that we want the rows corresponding to the top 10 values of <code>DepthRangeDeep</code>.</p>
<div class="sourceCode" id="cb61"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb61-1"><a href="3-wrangling.html#cb61-1" aria-hidden="true" tabindex="-1"></a>all_fishdata <span class="sc">%>%</span> <span class="fu">slice_max</span>(<span class="at">n =</span> <span class="dv">10</span>, <span class="at">order_by =</span> DepthRangeDeep)</span></code></pre></div>
<p>See the help file for <code>slice()</code> by running <code>?slice</code> for more information about its related functions.</p>
<div class="learncheck">
<p>
<strong><em>Learning check</em></strong>
</p>
</div>
<p><strong>(LC3.2)</strong> Repeat the previous command substituting the function <code>slice_head</code> for <code>slice_max</code>. How does the output differ?</p>
<p><strong>(LC3.3)</strong> Create a new data frame <code>notDeep_fish</code> that shows the rows of the <code>all_fishdata</code> data frame with the 20 smallest values of the <code>DepthRangeDeep</code> variable. (Check the <code>slice()</code> help file for hints.)</p>
<div class="learncheck">
</div>
</div>
<div id="select" class="section level2" number="3.4">
<h2><span class="header-section-number">3.4</span> <code>select</code> variables</h2>
<p>We recommended that you consider applying the <code>filter()</code> function to your data to narrow down the scope of your data frame to just the observations you care about. It may also be the case that you are only interested in a subset of the variables in your dataset. For example, the <code>all_fishdata</code> data frame has 101 variables, but typically only a few variables will be of interest for a particular analysis. You can identify the names of these 101 variables by running the <code>glimpse()</code> function from the <code>dplyr</code> package:</p>
<div class="sourceCode" id="cb62"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb62-1"><a href="3-wrangling.html#cb62-1" aria-hidden="true" tabindex="-1"></a><span class="fu">glimpse</span>(all_fishdata)</span></code></pre></div>
<p>In the same way that <code>filter</code> and <code>slice</code> return a subset of rows, the <code>select</code> function and its selection helpers allow us to return a subset of columns from a data frame.</p>
<div class="figure" style="text-align: center"><span id="fig:selectfig"></span>
<img src="images/cheatsheets/select.png" alt="Diagram of select() columns." width="\textwidth" />
<p class="caption">
FIGURE 3.2: Diagram of select() columns.
</p>
</div>
<p>Returning to <code>danger_comm_fish</code>, our data frame with dangerous, commercial, aquarium fish species, we might only really be interested in the variables <code>Species</code>, <code>Aquarium</code>, and <code>Dangerous</code>. However, with the current data frame, it’s very difficult to find these columns among all the others:</p>
<div class="sourceCode" id="cb63"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb63-1"><a href="3-wrangling.html#cb63-1" aria-hidden="true" tabindex="-1"></a><span class="fu">View</span>(danger_comm_fish)</span></code></pre></div>
<p>Examining these columns is much easier if we work with a smaller data frame by <code>select()</code>ing the desired variables:</p>
<div class="sourceCode" id="cb64"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb64-1"><a href="3-wrangling.html#cb64-1" aria-hidden="true" tabindex="-1"></a>slim_danger_comm_fish <span class="ot"><-</span> danger_comm_fish <span class="sc">%>%</span> </span>
<span id="cb64-2"><a href="3-wrangling.html#cb64-2" aria-hidden="true" tabindex="-1"></a> <span class="fu">select</span>(Species, Genus, Aquarium, Dangerous)</span></code></pre></div>
<p>This slimmer data frame makes it easy to verify that we correctly <code>filtered</code> for <code>commercial</code> and <code>highly commercial</code> species and also helps us to see the various ways that fish are not “harmless”:</p>
<div class="sourceCode" id="cb65"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb65-1"><a href="3-wrangling.html#cb65-1" aria-hidden="true" tabindex="-1"></a>slim_danger_comm_fish</span></code></pre></div>
<pre><code># A tibble: 229 x 4
Species Genus Aquarium Dangerous
<chr> <chr> <chr> <chr>
1 Ablabys taenianotus Ablabys commerci… venomous
2 Abudefduf vaigiensis Abudefduf commerci… reports of ciguatera pois…
3 Acanthodoras spinosissim… Acanthodoras commerci… venomous
4 Acanthostracion polygoni… Acanthostraci… commerci… reports of ciguatera pois…
5 Acanthostracion quadrico… Acanthostraci… commerci… reports of ciguatera pois…
6 Acanthurus achilles Acanthurus commerci… traumatogenic
7 Acanthurus chirurgus Acanthurus commerci… traumatogenic
8 Acanthurus coeruleus Acanthurus commerci… traumatogenic
9 Acanthurus leucopareius Acanthurus commerci… reports of ciguatera pois…
10 Acanthurus lineatus Acanthurus commerci… venomous
# … with 219 more rows</code></pre>
<p>Let’s say instead you want to drop, or de-select, certain variables. For example, it’s apparent that the information in the <code>Genus</code> variable is already present in the <code>Species</code> variable, so we may want to remove the <code>Genus</code> variable. We can deselect <code>Genus</code> by using the <code>-</code> sign:</p>
<div class="sourceCode" id="cb67"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb67-1"><a href="3-wrangling.html#cb67-1" aria-hidden="true" tabindex="-1"></a>slim_danger_comm_fish <span class="ot"><-</span> slim_danger_comm_fish <span class="sc">%>%</span> <span class="fu">select</span>(<span class="sc">-</span>Genus)</span></code></pre></div>
<p>Another way of selecting columns/variables is by specifying a range of columns. For example, we might want to know which of the <code>danger_comm_fish</code> species live in fresh, brackish, or saltwater environments:</p>
<div class="sourceCode" id="cb68"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb68-1"><a href="3-wrangling.html#cb68-1" aria-hidden="true" tabindex="-1"></a>slim2_danger_comm_fish <span class="ot"><-</span> danger_comm_fish <span class="sc">%>%</span> <span class="fu">select</span>(SpecCode<span class="sc">:</span>Species, Fresh<span class="sc">:</span>Saltwater)</span>
<span id="cb68-2"><a href="3-wrangling.html#cb68-2" aria-hidden="true" tabindex="-1"></a>slim2_danger_comm_fish</span></code></pre></div>
<pre><code># A tibble: 229 x 5
SpecCode Species Fresh Brack Saltwater
<dbl> <chr> <dbl> <dbl> <dbl>
1 10232 Ablabys taenianotus 0 0 -1
2 6630 Abudefduf vaigiensis 0 0 -1
3 12103 Acanthodoras spinosissimus -1 0 0
4 4287 Acanthostracion polygonius 0 0 -1
5 92 Acanthostracion quadricornis 0 0 -1
6 4306 Acanthurus achilles 0 0 -1
7 943 Acanthurus chirurgus 0 0 -1
8 944 Acanthurus coeruleus 0 0 -1
9 4737 Acanthurus leucopareius 0 0 -1
10 1258 Acanthurus lineatus 0 0 -1
# … with 219 more rows</code></pre>
<p>The <code>select()</code> function can also be used to reorder columns when combined with the <code>everything()</code> helper function. For example, suppose we want the <code>Species</code>, <code>BodyShapeI</code>, <code>DepthRangeShallow</code> and <code>DepthRangeDeep</code> variables to appear first, while not discarding the rest of the variables. In the following code, <code>everything()</code> will pick up all remaining variables:</p>
<div class="sourceCode" id="cb70"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb70-1"><a href="3-wrangling.html#cb70-1" aria-hidden="true" tabindex="-1"></a>fishdata_reorder <span class="ot"><-</span> all_fishdata <span class="sc">%>%</span> </span>
<span id="cb70-2"><a href="3-wrangling.html#cb70-2" aria-hidden="true" tabindex="-1"></a> <span class="fu">select</span>(Species, BodyShapeI, DepthRangeShallow, DepthRangeDeep, <span class="fu">everything</span>())</span>
<span id="cb70-3"><a href="3-wrangling.html#cb70-3" aria-hidden="true" tabindex="-1"></a><span class="fu">glimpse</span>(fishdata_reorder)</span></code></pre></div>
<p>See the help file for <code>select()</code> by running <code>?select</code> for more information about other selection helpers.</p>
<div class="learncheck">
<p>
<strong><em>Learning check</em></strong>
</p>
</div>
<p><strong>(LC3.4)</strong> Run the code <code>all_fishdata %>% select(starts_with("Depth"))</code> to select columns with names that start with “Depth.” How many columns are returned?</p>
<p><strong>(LC3.5)</strong> Use the <code>contains()</code> helper function to select columns from the <code>all_fishdata</code> data frame that contain “Pic.” How many columns are returned?</p>
<p><strong>(LC3.6)</strong> What if you forgot to include the double-quotes for the first command above? Run the code <code>all_fishdata %>% select(starts_with(Depth))</code> to see what happens.</p>
<div class="learncheck">
</div>
<p>The last command shows an example of what happens when you forget to include double-quotes.
If you see an <code>Error</code> message about an <code>object ... not found</code>, try adding double-quotes to see if that fixes the problem.</p>
<div id="rename" class="section level3" number="3.4.1">
<h3><span class="header-section-number">3.4.1</span> <code>rename</code> variables</h3>
<p>Another useful function similar to <code>select()</code> is <code>rename()</code>, which as you may have guessed changes the name of specified variables. Suppose we want to rename the <code>DemersPelag</code> variable to something more understandable, such as <code>Preferred_Habitat</code>:</p>
<div class="sourceCode" id="cb71"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb71-1"><a href="3-wrangling.html#cb71-1" aria-hidden="true" tabindex="-1"></a>all_fishdata <span class="sc">%>%</span> </span>
<span id="cb71-2"><a href="3-wrangling.html#cb71-2" aria-hidden="true" tabindex="-1"></a> <span class="fu">rename</span>(<span class="at">Preferred_Habitat =</span> DemersPelag)</span></code></pre></div>
<p>Only the name of the specified variable has changed, and all of the other variables remain intact and unchanged. Note that here we used a single <code>=</code> sign within the <code>rename()</code>. For example, <code>Preferred_Habitat = DemersPelag</code> renames the <code>DemersPelag</code> variable to have the new name <code>Preferred_Habitat</code>. This is because we are not testing for equality like we would using <code>==</code>. Instead we want to assign a new variable <code>Preferred_Habitat</code> to have the same values as <code>DemersPelag</code> and then delete the variable <code>DemersPelag</code>.</p>
<p>Tip: New <code>dplyr</code> users often forget that the new variable name comes before the equal sign, followed by the old variable. You can remember this as “New Before Old.” Tip 2: Avoid spaces and special symbols in your variable names, which in our experience can cause problems in R.</p>
<p>Pro-tip: We can also rename variables as we <code>select()</code> them from a data frame:</p>
<div class="sourceCode" id="cb72"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb72-1"><a href="3-wrangling.html#cb72-1" aria-hidden="true" tabindex="-1"></a>all_fishdata <span class="sc">%>%</span> </span>
<span id="cb72-2"><a href="3-wrangling.html#cb72-2" aria-hidden="true" tabindex="-1"></a> <span class="fu">select</span>(Species, <span class="at">Preferred_Habitat =</span> DemersPelag)</span></code></pre></div>
<pre><code># A tibble: 34,299 x 2
Species Preferred_Habitat
<chr> <chr>
1 Aapticheilichthys websteri benthopelagic
2 Aaptosyax grypus pelagic
3 Abactochromis labrosus demersal
4 Abalistes filamentosus pelagic-neritic
5 Abalistes stellaris demersal
6 Abalistes stellatus reef-associated
7 Abbottina binhi benthopelagic
8 Abbottina lalinensis benthopelagic
9 Abbottina liaoningensis demersal
10 Abbottina obtusirostris benthopelagic
# … with 34,289 more rows</code></pre>
<p>Here we selected two columns from <code>all_fishdata</code>, changing the name of the second one in the process.</p>
</div>
</div>
<div id="summarize" class="section level2" number="3.5">
<h2><span class="header-section-number">3.5</span> <code>summarize</code> variables</h2>
<p>Another common task when working with data frames is to compute <em>summary statistics</em>. Summary statistics are single numerical values that summarize a large number of values. Commonly known examples of summary statistics include the mean (also called the average) and the median (the middle value). Other examples of summary statistics that might not immediately come to mind include the <em>sum</em>, the smallest value also called the <em>minimum</em>, the largest value also called the <em>maximum</em>, and the <em>standard deviation</em>, a measure of the variability or “spread” in the values. See Appendix <a href="A-appendixA.html#appendix-stat-terms">A.1</a> for a glossary of such summary statistics.</p>
<p>Let’s calculate two summary statistics of the <code>Length</code> variable in the <code>brackish_fish</code> data frame: the mean and standard deviation. To compute these summary statistics, we need the <code>mean()</code> and <code>sd()</code> <em>summary functions</em> in R. Summary functions in R take in many values and return a single value, as illustrated in Figure <a href="3-wrangling.html#fig:summary-function">3.3</a>.</p>
<div class="figure" style="text-align: center"><span id="fig:summary-function"></span>
<img src="images/cheatsheets/summary.png" alt="Diagram illustrating a summary function in R." width="\textwidth" />
<p class="caption">
FIGURE 3.3: Diagram illustrating a summary function in R.
</p>
</div>
<p>More precisely, we’ll use the <code>mean()</code> and <code>sd()</code> summary functions within the <code>summarize()</code> function from the <code>dplyr</code> package. Note you can also use the British English spelling of <code>summarise()</code>. As shown in Figure <a href="3-wrangling.html#fig:sum1">3.4</a>, the <code>summarize()</code> function takes in a data frame and returns a data frame with only one row corresponding to the summary statistics.</p>
<div class="figure" style="text-align: center"><span id="fig:sum1"></span>
<img src="images/cheatsheets/summarize1.png" alt="Diagram of summarize() rows." width="80%" height="80%" />
<p class="caption">
FIGURE 3.4: Diagram of summarize() rows.
</p>
</div>
<p>We’ll save the results in a new data frame called <code>summary_brackish</code> that will have two columns/variables: the <code>mean</code> and the <code>std_dev</code>:</p>
<div class="sourceCode" id="cb74"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb74-1"><a href="3-wrangling.html#cb74-1" aria-hidden="true" tabindex="-1"></a>summary_brackish <span class="ot"><-</span> brackish_fish <span class="sc">%>%</span> </span>
<span id="cb74-2"><a href="3-wrangling.html#cb74-2" aria-hidden="true" tabindex="-1"></a> <span class="fu">summarize</span>(<span class="at">mean =</span> <span class="fu">mean</span>(Length), <span class="at">std_dev =</span> <span class="fu">sd</span>(Length))</span>
<span id="cb74-3"><a href="3-wrangling.html#cb74-3" aria-hidden="true" tabindex="-1"></a>summary_brackish</span></code></pre></div>
<pre><code># A tibble: 1 x 2
mean std_dev
<dbl> <dbl>
1 NA NA</code></pre>
<p>Why are the values returned <code>NA</code>? As we saw in Subsection <a href="2-viz.html#geompoint">2.3.1</a> when creating the scatterplot of lengths and weights for <code>brackish_fish</code>, <code>NA</code> is how R encodes <em>missing values</em> where <code>NA</code> indicates “not available” or “not applicable.” If a value for a particular row and a particular column does not exist, <code>NA</code> is stored instead. Values can be missing for many reasons. Perhaps the data was collected but someone forgot to enter it? Perhaps the data was not collected at all because it was too difficult to do so? Perhaps there was an erroneous value that someone entered that has been corrected to read as missing? You’ll often encounter issues with missing values when working with real data.</p>
<p>Going back to our <code>summary_brackish</code> output, by default any time you try to calculate a summary statistic of a variable that has one or more <code>NA</code> missing values in R, <code>NA</code> is returned. To work around this fact, you can set the <code>na.rm</code> argument to <code>TRUE</code>, where <code>rm</code> is short for “remove”; this will ignore any <code>NA</code> missing values and only return the summary value for all non-missing values.</p>
<p>The code that follows computes the mean and standard deviation of all non-missing values of <code>Length</code>:</p>
<div class="sourceCode" id="cb76"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb76-1"><a href="3-wrangling.html#cb76-1" aria-hidden="true" tabindex="-1"></a>summary_brackish <span class="ot"><-</span> brackish_fish <span class="sc">%>%</span> </span>
<span id="cb76-2"><a href="3-wrangling.html#cb76-2" aria-hidden="true" tabindex="-1"></a> <span class="fu">summarize</span>(<span class="at">mean =</span> <span class="fu">mean</span>(Length, <span class="at">na.rm =</span> <span class="cn">TRUE</span>), </span>
<span id="cb76-3"><a href="3-wrangling.html#cb76-3" aria-hidden="true" tabindex="-1"></a> <span class="at">std_dev =</span> <span class="fu">sd</span>(Length, <span class="at">na.rm =</span> <span class="cn">TRUE</span>))</span>
<span id="cb76-4"><a href="3-wrangling.html#cb76-4" aria-hidden="true" tabindex="-1"></a>summary_brackish</span></code></pre></div>
<pre><code># A tibble: 1 x 2
mean std_dev
<dbl> <dbl>
1 45.7 69.7</code></pre>
<p>Notice how the <code>na.rm = TRUE</code> are used as arguments to the <code>mean()</code> and <code>sd()</code> summary functions individually, and not to the <code>summarize()</code> function.</p>
<p>However, one needs to be cautious whenever ignoring missing values as we’ve just done. In the upcoming <em>Learning checks</em> questions, we’ll consider the possible ramifications of blindly sweeping rows with missing values “under the rug.” This is in fact why the <code>na.rm</code> argument to any summary statistic function in R is set to <code>FALSE</code> by default. In other words, R does not ignore rows with missing values by default. R is alerting you to the presence of missing data and you should be mindful of this absence and any potential causes of this absence throughout your analysis.</p>
<p>What are other summary functions we can use inside the <code>summarize()</code> verb to compute summary statistics? As seen in the diagram in Figure <a href="3-wrangling.html#fig:summary-function">3.3</a>, you can use any function in R that takes many values and returns just one. Here are just a few:</p>
<ul>
<li><code>mean()</code>: the average</li>
<li><code>sd()</code>: the standard deviation, which is a measure of spread</li>
<li><code>min()</code> and <code>max()</code>: the minimum and maximum values, respectively</li>
<li><code>IQR()</code>: interquartile range</li>
<li><code>sum()</code>: the total amount when adding multiple numbers</li>
<li><code>n()</code>: a count of the number of rows in each group. This particular summary function will make more sense when <code>group_by()</code> is covered in Section <a href="3-wrangling.html#groupby">3.6</a>.</li>
</ul>
<div class="learncheck">
<p>
<strong><em>Learning check</em></strong>
</p>
</div>
<p><strong>(LC3.7)</strong> Say a doctor is studying the effect of smoking on lung cancer for a large number of patients who have records measured at five-year intervals. She notices that a large number of patients have missing data points because the patient has died, so she chooses to ignore these patients in her analysis. What is wrong with this doctor’s approach?</p>
<p><strong>(LC3.8)</strong> Modify the earlier <code>summarize()</code> function code that creates the <code>summary_brackish</code> data frame to also use the <code>n()</code> summary function: <code>summarize(... , count = n())</code>. What does the returned value correspond to?</p>
<p><strong>(LC3.9)</strong> Why doesn’t the following code work? Run the code line-by-line instead of all at once, and then look at the data. In other words, run <code>summary2_brackish <- brackish_fish %>% summarize(mean = mean(Length, na.rm = TRUE))</code> first.</p>
<div class="sourceCode" id="cb78"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb78-1"><a href="3-wrangling.html#cb78-1" aria-hidden="true" tabindex="-1"></a>summary_wt_d21 <span class="ot"><-</span> brackish_fish <span class="sc">%>%</span> </span>
<span id="cb78-2"><a href="3-wrangling.html#cb78-2" aria-hidden="true" tabindex="-1"></a> <span class="fu">summarize</span>(<span class="at">mean =</span> <span class="fu">mean</span>(Length, <span class="at">na.rm =</span> <span class="cn">TRUE</span>)) <span class="sc">%>%</span> </span>
<span id="cb78-3"><a href="3-wrangling.html#cb78-3" aria-hidden="true" tabindex="-1"></a> <span class="fu">summarize</span>(<span class="at">std_dev =</span> <span class="fu">sd</span>(Length, <span class="at">na.rm =</span> <span class="cn">TRUE</span>))</span></code></pre></div>
<div class="learncheck">
</div>
</div>
<div id="groupby" class="section level2" number="3.6">
<h2><span class="header-section-number">3.6</span> <code>group_by</code> rows</h2>
<!-- To get `_` to work in caption title. Found at https://github.com/rstudio/bookdown/issues/209 -->
<div class="figure" style="text-align: center"><span id="fig:groupsummarize"></span>
<img src="images/cheatsheets/group_summary.png" alt="Diagram of group_by() and summarize()." width="\textwidth" />
<p class="caption">
FIGURE 3.5: Diagram of group_by() and summarize().
</p>
</div>
<p>Above we calculated the mean <code>Length</code> of <code>brackish_fish</code>. Say instead of a single mean <code>Length</code> for a dataset, we would like to compute the mean lengths of fish species in different habitats separately, that is, the mean length split by preferred habitats. We can do this by “grouping” the <code>Length</code> observations by the values of another variable, in this case by the values of the variable <code>DemersPelag</code>. Run the following code:</p>
<!--
New dplyr warning message when running group_by() %>% summarize() that is not
addressed in v1 (print edition).
See https://github.com/moderndive/ModernDive_book/issues/353
For now we suppress this message in the book by setting
options(dplyr.summarise.inform = FALSE) in index.Rmd
v2 TODO: Address this warning message explicitly in text and fix index.Rmd
-->
<div class="sourceCode" id="cb79"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb79-1"><a href="3-wrangling.html#cb79-1" aria-hidden="true" tabindex="-1"></a>summary_DP_length <span class="ot"><-</span> all_fishdata <span class="sc">%>%</span> </span>