draft 2 slides

Author: libjohn

README.Rmd

@@ -21,10 +21,7 @@ knitr::opts_chunk$set(
 ### Install Packages
 
 ```
-install.packages("janeaustenr")
-install.packages("tidyverse")
-install.packages("tidytext")
-install.packages("wordcloud2")
+install.packages(c("tidyverse", "tidytext", "janeaustenr", "wordcloud2"))
 ```
 ### Process
 

README.md

@@ -1,21 +1,17 @@
 README
 ================
-2020-10-30
+2021-04-13
 
 <!-- README.md is generated from README.Rmd. Please edit that file -->
 
 ## Text Mining: Senitment analysis and word clouds
 
 <!-- badges: start -->
-
 <!-- badges: end -->
 
 ### Install Packages
 
-    install.packages("janeaustenr")
-    install.packages("tidyverse")
-    install.packages("tidytext")
-    install.packages("wordcloud2")
+    install.packages(c("tidyverse", "tidytext", "janeaustenr", "wordcloud2"))
 
 ### Process
 
@@ -33,14 +29,14 @@ README
 
 ### Resources
 
-  - [Tidytext package](https://juliasilge.github.io/tidytext/)
-  - Book: [Text Mining with R](https://www.tidytextmining.com/) by Silge
+-   [Tidytext package](https://juliasilge.github.io/tidytext/)
+-   Book: [Text Mining with R](https://www.tidytextmining.com/) by Silge
     and Robinson
-  - Data Wrangling with dplyr:
-    ([video](https://juliasilge.github.io/tidytext/) |
+-   Data Wrangling with dplyr:
+    ([video](https://juliasilge.github.io/tidytext/) \|
     [workshop](https://rfun.library.duke.edu/portfolio/r_flipped/))
-  - Data Visualization with ggplot2:
-    ([video](https://warpwire.duke.edu/w/80YEAA/) |
+-   Data Visualization with ggplot2:
+    ([video](https://warpwire.duke.edu/w/80YEAA/) \|
     [workshop](https://rfun.library.duke.edu/portfolio/ggplot_workshop/))
 
 ![Word Cloud](images/word_cloud.PNG "Word Cloud")

slides/images/tidy1.svg

@@ -1,5 +1,5 @@
 ---
-title: "<br>Text Mining"
+title: "<br>Sentiment Analysis"
 subtitle: "R case study"
 author: "John Little"
 institute: "Cntr for Data & Viz"
@@ -38,9 +38,25 @@ tagList(rmarkdown::html_dependency_font_awesome())
 # )
 ```
 
+## Packages for today
+
+_Sentiment Analysis:  R case study_  
+
+.bg-washed-blue.b--navy.l-10.t-20pct.w-90.ba.bw2.br3.shadow-5.ph4.mt5[
+`install.packages(c("tidyverse", "tidytext",`<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;`"janeaustenr", "wordcloud2"))`
+]  
+
+.l-60.t-80pct.fl-w-third[
+```{r echo=FALSE}
+countdown::countdown(20)
+```
+]
+
+---
+
 ## Duke University: Land Acknowledgement
 
-.f4[I would like to take a moment to honor the land in Durham, NC.  Duke University sits on the ancestral lands of the Shakori, Eno and Catawba people. This institution of higher education is built on land stolen from those peoples.  These tribes were here before the colonizers arrived.  Additionally this land has borne witness to over 400 years of the enslavement, torture, and systematic mistreatment of African people and their descendants.  Recognizing this history is an honest attempt to breakout beyond persistent patterns of colonization and to rewrite the erasure of Indigenous and Black peoples.  There is value in acknowledging the history of our occupied spaces and places.  I hope we can glimpse an understanding of these histories by recognizing the origins of collective journeys.]
+.f3[I would like to take a moment to honor the land in Durham, NC.  Duke University sits on the ancestral lands of the Shakori, Eno and Catawba people. This institution of higher education is built on land stolen from those peoples.  These tribes were here before the colonizers arrived.  Additionally this land has borne witness to over 400 years of the enslavement, torture, and systematic mistreatment of African people and their descendants.  Recognizing this history is an honest attempt to breakout beyond persistent patterns of colonization and to rewrite the erasure of Indigenous and Black peoples.  There is value in acknowledging the history of our occupied spaces and places.  I hope we can glimpse an understanding of these histories by recognizing the origins of collective journeys.]
 
 
 ---
@@ -56,196 +72,173 @@ layout: true
 
 ## Demonstration Goals
 
-- Gather some tweets  
+- Data _cleaning_ & data _wrangling_
+
+- Tokenize corpora (unit of analysis)
 
-- Define APIs and the Twitter Developer portal (Academic Use)  
+- Visualize word clouds (novelty)  
 
-- Rudimentary text analysis and visualization  
+- Sentiment analysis ()
 
-- Point out useful documentation / resources  
+- Analyzing word frequencies (tf-idf)
 
 
 ***
 
 .f6.i.moon-gray.center[This is not a text analysis workshop.  The foundations of text analysis require considerably more time that we have.  
-This is a demonstration on leveraging the following tidy packages (tidyverse, and  tidytext) and sharing resources.  ]
+This is a demonstration on leveraging tidy packages (tidyverse and tidytext) and sharing resources.  ]
 
 
 ---
 
 class: img-right-full
 
-![](images/attendance.png)
+# _Text Mining with R_ 
 
-# Three tenets
+![](https://www.tidytextmining.com/images/cover.png)
 
+#### by Silge & Robinson 
 
-- Just numbers
-- Benefits of review
-- Dashboard fatigue is a real thing 
 
+- [www.tidytextmining.com](https://www.tidytextmining.com/)  
 
-???
+- [juliasilge.github.io/tidytext](https://juliasilge.github.io/tidytext/)  
+
+- [github.com/juliasilge/janeaustenr](https://github.com/juliasilge/janeaustenr)
 
-- The implications of dashboard fatigue might be the most interesting thing to discuss in the QA
 
 ---
-layout: false
-class: img-left-full
 
-![](images/by_dept_compare.png)
+class: img-left-full  
+layout: false  
 
-## Drivers
+![](images/tidy1.svg)
 
-- Goal: create a dashboard of workshop attendance 
-- CDVS motivated by the possibility of exploring data 
-- Dashboard can be the basis of another  workshop 
+# Tidy data
 
-.footercc[
-<i class="fab fa-creative-commons"></i>&nbsp; <i class="fab fa-creative-commons-by"></i><i class="fab fa-creative-commons-nc"></i>  <a href = "https://JohnLittle.info"><span class = "opacity30">https://</span>JohnLittle<span class = "opacity30">.info</span></a> 
-<span class = "opacity30"> | <a href="https://github.com/libjohn/workshop_textmining">https://github.com/libjohn/workshop_textmining</a> | `r Sys.Date()` </span>
-]
+- Each variable is a column  
 
-???
+- Each observation is a row  
 
-- These are not exactly the best drivers for creating a dashboard.  They’re not bad either.
+- Each type of observational unit is a table  
 
+.footer.center[.f6[-- Wickham 2014  ]  ]
 
----
-layout: false
-class:  middle, center
+.footercc[  [Tidy data. Chapter 12. _R for Data Science_](https://r4ds.had.co.nz/tidy-data.html) by Wickham & Grolemund]
 
-<br>
+---
 
-.bg-washed-blue.b--navy.ba.bw2.br3.shadow-5.ph4.mt5[
+class: img-right-full  
 
-![Rfun](images/rfun.png# fl l-4 w-2-12th)
+![](images/tidy2.svg)
 
-## John R Little
+# Tidy Text format
 
-.prussian[
-.f5[Data Science Librarian  
-Center for Data & Visualization Sciences  
-Duke University Libraries  
-]
-]
+- A token is a meaningful unit of text
 
-.f7[https://johnlittle.info  
-https://Rfun.library.duke.edu  
-https://library.duke.edu/data  
-]
-]
+- Tokenization is the process of splitting text into tokens `tidytext::unnest_tokens()`
 
+- A table with **one-token-per-row**
 
+.footer[.f6[  &nbsp; -- Silge & Robinson] ]
 
-<i class="fab fa-creative-commons fa-2x"></i> &nbsp; <i class="fab fa-creative-commons-by fa-2x"></i><i class="fab fa-creative-commons-nc fa-2x"></i>  
-.f6.moon-gray[Creative Commons:  Attribution-NonCommercial 4.0]  
-.f7.moon-gray[https://creativecommons.org/licenses/by-nc/4.0]
 
 ---
-class: inverse
+class: col-2  
 
-# Appendix
+# Other data structures  
 
-## screen shots
+#### String  
+
+Text / character vectors
+
+
+#### Corpus  
+Raw strings annotated with additional metadata  
+
+#### Document-term matrix
+
+A sparse matrix describing a collection of documents (i.e. _corpus_) with one row for each document and one column for each term. (tf-idf)
+
+
+.footer[.f6[ &nbsp; -- Silge & Robinson]] 
 
 ---
+
 layout: true
 
 .footercc[
 <i class="fab fa-creative-commons"></i>&nbsp; <i class="fab fa-creative-commons-by"></i><i class="fab fa-creative-commons-nc"></i>  <a href = "https://JohnLittle.info"><span class = "opacity30">https://</span>JohnLittle<span class = "opacity30">.info</span></a> 
 <span class = "opacity30"> | <a href="https://github.com/libjohn/workshop_textmining">https://github.com/libjohn/workshop_textmining</a> | `r Sys.Date()` </span>
 ]
 
----
-
-![Tidyverse](images/tidyverse.png# w-10pct t-1 db fr mr-4)
-
-## Technology stack 
 
+---
 
+# Other packages <sup>✤</sup>
 
-- R   
-    - R is a data-first coding language   
-    - R can be a universal interface for analysis and workflow   
-- Tidyverse is a well developed approach to workflow & the data lifecycle  
-- Bias towards enabling reproducibility   
-    - scripting   
-    - reporting   
-    
+- [tm](https://tm.r-forge.r-project.org/) -- _Text Mining Infrastructure in R_  
 
-![flexdashboards](images/flexdashboard.png#  w-10pct fr fm mr-4)  
+- [quanteda](https://quanteda.io/) -- _Package for managing and analyzing textual data_
 
-![r logo](images/r_logo.png# fm fr w-10pct mr-4)  
+- [gutenbergr](https://docs.ropensci.org/gutenbergr/) -- public domain text from **Project Gutenberg**
 
-![rmarkdown](images/rmarkdown.png#  w-10pct fr mr-4)  
 
-    
-???
+.footnote[.small[ ✤ Not covered in this case study]]
 
-- Reuse analysis code to produce reports, email alerts, interactive dashboards, etc.  
 
 ---
 
-## Lesson
+# Further study
 
-.fl-10.w-60.bg.b.ba.bw1.br3.shadow-5.ph4.mt4.center.prussian[The last thing you should do is  
-build the dashboard  
-]  
+Read more of [_Text Mining with R: A Tidy Approach_](https://www.tidytextmining.com)
 
-- Identify target audience and scope  
-- Create summary reports   
-- Build a static analysis   
-- Generate push-reports based on dynamic thresholds  
-- Advanced: Build a reporting application
+1. The tidy text format
+2. **Sentiment analysis with tidy data**
+3. **Analyzing word and document frequency: tf-idf**
+4. Relationships between words: n-grams and correlations
+5. Converting to and from non-tidy formats
+6. **Topic modeling** (unsupervised classification)
+7. Case study: comparing Twitter archives<br>_plus more case studies_
 
-???
-Or, in this case, build a workshop attendance application
 
 ---
-## Other important question(s) 
 
-- If developing the dashboard in R...  
-    - Flexdashboard (dashboards)
-    - Shiny (Web applications)
+# Further study
 
-Not mutually exclusive but Flexdashboards has a significantly lower barrier to entry  
+_Summer Institute for Computational Social Science_  
+co-founded by [Chris Bail & Matthew Salganik](https://sicss.io/people)
 
-.center[![people](images/happy_people2.jpg# h-10pct w-33pct)] 
+[SICSS Text Analysis curriculum](https://sicss.io/curriculum)
 
 ---
-## Actual Goals  
-
-- Host **cleaned and disaggregated data** 
-
-- Provide a **summary of attendance** 
-
-![survey](images/survey_1.png# absolute ofv r-3 w-75pct h-7-12th)
-
+layout: false
+class:  middle, center
 
-    
-???
+<br>
 
-- Host **cleaned and disaggregated data** 
-    - A data archive for clean data 
-        - exported from the SpringShare registration system 
-        - accounts for attendance
-- Provide a **summary of attendance** so that staff can 
-    - Assess their workshop’s impact over time (as measured by attendance and registration)
-    - See current semester attendance totals within the context of multi-year totals
+.bg-washed-blue.b--navy.ba.bw2.br3.shadow-5.ph4.mt5[
 
----
-class: center
+![Rfun](images/rfun.png# fl l-4 w-2-12th)
 
-![](images/full_attendance.png# l-0 t-0 w-two-thirds h-80pct ofv absolute)
-![](images/full_demographics.png# w-third h-40pct t-0 r-0 ofv absolute)
-![](images/full_survey.png# w-third h-40pct t-40pct r-0 ofv absolute)
-![](images/slice_tables.jpg# l-0 t-80pct w-100pct ofv absolute)
+## John R Little
 
 .prussian[
-.absolute.w-5-12th.pa-3.l-4-12th.t-8-12th.b.ba.bw-4.br-4.shadow-5.bg-white-80[
-Collage of dashboard screens
+.f5[Data Science Librarian  
+Center for Data & Visualization Sciences  
+Duke University Libraries  
 ]
 ]
 
+.f7[https://johnlittle.info  
+https://Rfun.library.duke.edu  
+https://library.duke.edu/data  
+]
+]
+
+
+
+<i class="fab fa-creative-commons fa-2x"></i> &nbsp; <i class="fab fa-creative-commons-by fa-2x"></i><i class="fab fa-creative-commons-nc fa-2x"></i>  
+.f6.moon-gray[Creative Commons:  Attribution-NonCommercial 4.0]  
+.f7.moon-gray[https://creativecommons.org/licenses/by-nc/4.0]