Computational Communication Research: Inaugural Issue

We are very happy to announce the inaugural issue for Computational Communication Research! The articles are currently in production, but you can access the preprints using the links below.  Please help us spread the word!

We would like to thank all reviewers, submitters, and editorial board members for contributing to the journal and for their feedback on this introduction. We would also like to thank Amsterdam University Press and especially our gold sponsors (Vrije Universiteit Amsterdam, The Network Institute, the University of Amsterdam / ASCoR) and silver sponsors (The Hebrew University of Jerusalem, The Center for Information Technology and Society at UC Santa Barbara, and the Computational Communication Science Lab of the University of Vienna), for making this journal possible.

Looking forward to your submissions and reviews in the coming months!
CCR Inaugural Issue

Introduction: A Roadmap for Computational Communication Research [draft version]
Wouter van Atteveldt, Drew Margolin, Cuihua Shen, Damian Trilling, René Weber
[https://osf.io/preprints/socarxiv/4dhfk]

GDELT Interface for Communication Research (iCoRe)
Frederic R. Hopp, Jacob T. Fisher, René Weber
[https://osf.io/24n6a/]

An Experimental Study of Recommendation Algorithms for Tailored Health Communication
Hyun Suk Kim, Sijia Yang, Minji Kim, Brett Hemenway, Lyle Ungar, Joseph Cappella:
[https://osf.io/preprints/socarxiv/nu6tg]

News Organizations’ Selective Link Sharing as Gatekeeping: A Structural Topic Model Approach
Chankyung Pak
[https://osf.io/preprints/socarxiv/pt7es]

Computational observation: Challenges and opportunities of automated observation within algorithmically curated media environments using a browser plug-in
Mario Haim, Angela Nienierza
[https://osf.io/preprints/socarxiv/xd63n/]

Research talk University of Zurich

This afternoon I will give a research talk at the Institute of Communication and Media Research at the University of Zurich.

Building the Open Computational Communication Science toolchain

Computational Communication Science promises to give new insight into communication and social behavior by using digital methods to study large and heterogeneous data sets consisting of traces left by online activity from Instagram posts, comments to online news articles on various sites to online purchases. 
This talk focuses on the tools needed to carry out this research. In particular, we need tools to gather data, such as digital trace data; analyze the resulting texts, networks, and images to  measure our theoretical quantities; and store and share the data and results. In all cases, it is important to focus on the replicability, validity, and transparency of data, analytic processes, and results. In this talk, I will outline the requirements, existing resources and challenges for “open” Computational Communication Science.  For each of these steps, I will discuss the possibilities and limitations of existing tools, and describe the methods and open sources tools that we are currently developing. I will call for a turn to “open science” and collaboration on open source software to build the tools we need to develop Computational Communication Science.

Research talk in Vienna

This afternoon I will give a talk at the Research Colloquium of the Institut für Publizistik- und Kommunikationswissenschaft of the University of Vienna

Title: Building the Open Computational Communication Science toolchain

Abstract: Computational Communication Science promises to give new insight into communication and social behavior by using digital methods to study large and heterogeneous data sets consisting of traces left by online activity from Instagram posts, comments to online news articles on various sites to online purchases.
This talk focuses on the tools needed to carry out this research. In particular, we need tools to gather data, such as digital trace data; analyze the resulting texts, networks, and images to  measure our theoretical quantities; and store and share the data and results. In all cases, it is important to focus on the replicability, validity, and transparency of data, analytic processes, and results. In this talk, I will outline the requirements, existing resources and challenges for “open” Computational Communication Science.  For each of these steps, I will discuss the possibilities and limitations of existing tools, and describe the methods and open sources tools that we are currently developing. I will call for a turn to “open science” and collaboration on open source software to build the tools we need to develop Computational Communication Science.

Text Analysis in R workshop at U. Vienna

As part of my Paul Lazarsfeld Guest Professorship I will teach a workshop on text analysis in R at the University of Vienna from 8 – 12 April.

For participants: Please bring your own laptop and make sure you have R and RStudio installed.

Introduction: The explosion of digital communication and increasing efforts to digitize existing material has produced a deluge of material such as digitized historical news archives, policy and legal documents, political debates or millions of social media messages by politicians, journalists, and citizens. This has the potential of putting theoretical predictions about the societal roles played by information, and the development and effects of communication to rigorous quantitative tests that were impossible before. Besides providing an opportunity, the analysis of such “big data” sources also poses methodological challenges. Traditional manual content analysis does not scale to very large data sets due to high cost and complexity. For this reason, many researchers turn to automatic text analysis using techniques such as dictionary analysis, automatic clustering and scaling of latent traits, and machine learning.

Course aims and structure: To properly use such techniques, however, requires a very specific skillset. This course aims to give interested PhD (and advanced Master) students an introduction to text analysis. R will be used as platform and language of instruction, but the basic principles and methods are easily generalizable to other languages and tools such as python. Participants will be given handouts with examples based on pre-existing data to follow along, but are encouraged to work on their own data and problems using the techniques offered.

Evaluation criteria: Evaluation will be based on two assignments:

  1. (30%) midweek data exercise
    1. Deadline: Wednesday (soft)
    2. Instructions
    3. Data
    4. Submission link
  2. (70%) final assignment  on a topic of your choice
    1. Deadline: Friday 19 April
    2. Instructions
    3. Submission link

There’s also a Optional/formative quiz to test your tidyverse skills

Material: The course mostly uses the handouts linked below per session. The source code of the handouts is available on Github. Also see the rstudio cheat sheets and the excellent book R for Data Science.

Course outline per day (A=morning, B=afternoon):

  1. Monday: Introduction to R
    1. (  9:00-11:00)
      1. R Basics: data and functions (practise template);
      2. Fun with Text
    2. (14:00-16:00)
      1. Tidyverse: Transforming  data;
      2. reading and importing data (external tutorial)
  2. Tuesday: R for data analysis
    1. (  9:00-11:15)
      1. Grouping and summarizing data
      2. Merging (joining) data sets
    2. (13:30-16:00)
      1. Visualizing data with ggplot
      2. Reshaping data: wide, long, and tidy
  3. Wednesday: Quantitative text analysis in R
    1. (9:00-13:00)
      1. Basic string handling in R [session log – warning, might be messy!]
      2. Reading, cleaning, and analysing text with quanteda and readtext [messy session log]
  4. Thursday: Topic Modeling  and Preprocessing
    1. (  9:00-12:00)
      1. Topic Modeling [slides] [handout]
        Optional handouts: [graphical interpretation] [perplexity code]
      2. NLP Preprocessing [slides] [handout]
    2. (14:00-16:00)
      1. Understanding topic modeling (slides)
        optional links: [gibbs sampling in R][understanding alpha]
      2. Structural Topic Model [slides] [handout] [vignette]
  5. Friday: Supervised machine learning
    1. (  9:00-12:00) Supervised text classification [slides] [handout]
    2. (14:00-16:00) Work on assignment

Course Literature:

Kasper Welbers, Wouter van Atteveldt, and Ken Benoit (2017), Text Analysis in R. Communication Methods and Measures, 11 (4), 245-265, doi:10.1080/19312458.2017.1387238

Wickham, H., & Grolemund, G. (2016). R for data science: import, tidy, transform, visualize, and model data. O’Reilly Media, Inc. .

Background literature:
– Wouter van Atteveldt and Tai-Quan Peng (2018), When Communication Meets
Computation: Opportunities, Challenges, and Pitfalls in Computational Communication
Science, Communication Methods and Measures 12 (2-3), pp. 81-92.
– Chang, J., Gerrish, S., Wang, C., Boyd-Graber, J. L., & Blei, D. M. (2009). Reading tea leaves: How humans interpret topic models. In Advances in neural information
processing systems (pp. 288-296).
– Denny, M. J., & Spirling, A. (2018). Text preprocessing for unsupervised learning: why it matters, when it misleads, and what to do about it. Political Analysis, 26(2), 168-189.
– Grimmer, J., & Stewart, B. M. (2013). Text as data: The promise and pitfalls of automatic content analysis methods for political texts. Political analysis, 21(3), 267-297.
– Roberts, M. E., Stewart, B. M., Tingley, D., Lucas, C., Leder‐Luis, J., Gadarian, S. K., … & Rand, D. G. (2014). Structural Topic Models for Open‐Ended Survey Responses. American Journal of Political Science, 58(4), 1064-1082.
– Young, L., & Soroka, S. (2012). Affective news: The automated coding of sentiment in political texts. Political Communication, 29(2), 205-231.

–  Goldberg, Y. (2017). Neural network methods for natural language processing. Synthesis Lectures on Human Language Technologies10(1), 1-309. If you google for neural network methods for natural language processing pdf you might be able to find the evaluation sample from the publisher.

 

Etmaal 2019: Mobile tracking and crowd coding

My presentations for Etmaal (Dutch-Flemish communication science conference) 2019:

Gathering Mobile News Consumption Traces: An Overview of Possibilities and a Prototype Tool (not really based on Google Takeout)
Wouter Van Atteveldt, Laurens Bogaardt, Vincent van Hees, Felicia Loecherbach, Judith Moeller and Damian Trilling

Download [Poster][Slides]


Sentiment Analysis: what is great and what sucks?
Wouter van Atteveldt, Mariken van der Velden, Mark Boukes

Download [Slides]

Educational quality monitoring: Is it about quality or about monitoring?

(Note: as part of my ‘senior teacher qualification’ I was asked to write down my vision on an aspect of education. I wrote the essay below on the role of quality monitoring in  (higher) education. All feedback welcome!)

Abstract: There are many opportunities for using educational assessment and quality monitoring instruments to improve the quality of education. If these instruments are mainly seen by faculty as “feeding the bureaucratic beast”, however, it is quite likely that they will not contribute to a real quality culture, but rather cause a loss of perceived professional autonomy and frustration over time spent “ticking boxes” rather than preparing classes or giving feedback to students. I have provided a number of recommendations to achieve more constructive monitoring. In short, goals should be clearly and sincerely explained; measurement should be close to the substantive goals; monitoring should be minimally obtrusive; and faculty needs to be included as the main actor rather than as the object of monitoring. Following these recommendations may sometimes lead to fewer boxes being ticked, but will hopefully contribute to a real improvement in teaching quality and job satisfaction and productivity of teaching faculty. 

In the guise of enhancing professionalisation and accountability of teaching, a number of monitoring instruments have been introduced at universities in the Netherlands and abroad. Where professors used to be relatively free to teach and test ‘their’ courses as they saw fit, nowadays at the VU a course is encapsulated in an assessment plan outlining how the course fits into the different learning trajectories and which learning outcomes need to be tested in what way; after teaching the course the coordinator has to submit a course dossier showing (among others) how the tests were constructed and validated and (in the table of specifications or toetsmatrijs) how the elements of the test correspond to the various intended learning outcomes.

Now, do these monitoring instruments actually improve the quality of teaching, or whether they mostly hinder the teacher in doing his or her job, by adding extra work and diminishing professional autonomy and work pleasure. Put another way, the key question is: under what conditions do monitoring instruments contribute to the quality of education?

Why do we need quality monitoring?

The origins of increased monitoring and accountability lie in the adoption of a form of new public management, where a central authority sets the goals, and subordinate units have room to decide how to achieve these goals (e.g. Hoecht, 2006). Crucially, progress towards the goals needs to be measured (and hence measurable) to make the decentralized organization accountable.

There is generally no objection to accountability (or transparency) by itself, and there are many ways in which such accountability and monitoring can improve quality. The most obvious is perhaps that it can show which universities, programmes, or teachers are most successful at the various metrics related to their educational performance. This keeps faculty alert and motivated and gives incentives to adopt best practices from programmes or colleagues that are performing better.

It can also have direct beneficial effects on teaching. First, it can make teachers themselves aware of possible problems in their own teaching methods, and can point them towards resources or solutions. For example, the (in)famous table of specifications forces teachers to think about the proper distribution of tasks or questions of learning outcomes, and stimulates them to reflect on whether the test strategy adequately tests the outcome in question.

Second, by creating standards and improving documentation within the organization it can make it easier to transfer knowledge and experiences between teachers, for example when taking over a course or in an intervision setting. Students also benefit from more standardized information.

Potential problems with quality monitoring

However, it is not a given that all forms of monitoring and accountability lead to an improvement of education.  A central criticism of new public management is that by focussing on output control, and using certain metrics to measure the achievement of output, the focus of managers and indirectly of faculty moves from these “ultimate” goals towards more “proximate” measurable goals, such as presence of complete course files, graduation rates and student satisfaction. It can even shift from the actual output towards the measurement instrument, with the danger “valuing what is measured, rather than [measuring] what we value” (Biesta, 2009, p. 43).

Additionally, before we can determine how to measure outcomes (and hence the effectiveness of teaching as a process), we need to be able to clearly define outcomes. Although the goals of education are notoriously hard to define and measure, a useful analysis is the division of educational purposes in terms of qualification, socialization, and subjectivication (Biesta 2010; 2015), with most measurement targets aimed at the (arguably more concrete) goal of qualification.

A final criticism of new public management in education rests on the (implicit) assumption of students as customers, as evident in the important role of student evaluations in evaluating courses,  teachers, and even curricula (e.g. the Dutch NSE). Although students are certainly not blind to the quality of teaching, satisfaction ratings also tap into other variables such as enjoyment or strictness. Moreover, as Biesta (2015) describes, the relation between a student and teacher is closer to that between a doctor and a patient: the patient wishes to become better, but leaves it to the doctor to determine the treatment. Citing Feinberg (2001), Biesta states that professors (and doctors) “do not just service the needs of the client, but also play a crucial role in the definition of those needs” (2015, p. 82), especially on the purposes of socialization and subjectivication.

Partly because of measurement issues, and partly for cultural reasons, empirical research on the effectiveness of audits and measurements is scarce, but some anecdotal evidence is published. In an article titled “Feeding the Beast or Improving Quality?”, Jethro Newton (2010) studies whether the monitoring of educational quality, especially in the form of assessment exercises, contributes to education quality. Based on faculty interviews,  his main finding is that in many cases faculty sees accountability requirements as activities that do not contribute to the primary process, but are instead merely needed to ‘feed’ the bureaucratic ‘beast’. According to Newton, it is crucial for quality accountability to be ‘owned’ and ‘adopted’ by faculty for it to have a positive effect on teaching quality. Hoecht (2006) similarly conducted interviews at two institutions that recently attained university status and that went through a transition from ‘light-touch’ quality control to “a highly prescribed process of audit-based quality control” (p. 541). His conclusion is that, although “accountability and transparancy are important principles”, the current audit regime introduces “one-way accountability” and “rituals of verification” (p. 541; cf. Power, 1997), with interviewees commenting on the “extensive box-ticking at the expense of [..] activities such as teaching preparation” (p.556).

How to make quality monitoring beneficial?

From these considerations, it is clear that care must be taken on how quality monitoring and accountability are designed, implemented, and communicated. In my opinion, for quality monitoring to contribute to the quality of teaching, it is necessary for (1) the (substantive) goals of policy to be clear; (2) for the output measurement to be as close as possible to the substantive goals; (3) for the policy to be implemented in a way to maximize teaching quality; and (4) for faculty to have a feeling of ownership and recognition.

First, the goal of a specific policy must always be defined and communicated in substantive terms and related to improving the quality of teaching. The goal must also be sincerely defended by management. Requiring a specific action because “it is required by the visitation” is never an adequate response: management should either agree with and adopt the underlying goals of the policy, or resist its implementation and explain why the policy is not applicable or beneficial for this specific case.

Second, the attainment of policy goals should be measured as close as possible to the substantive goal. In many cases, this is not trivial. Learning outcomes are often difficult to measure, especially high-level educational goals such as socialization and subjectivication. As teaching is never done in isolation, the contribution of specific teaching activities to these outcomes is even more difficult to measure. As a consequence, we often take measures that are easy to produce, such as student satisfaction or graduation rates.

Third, the implementation of quality monitoring must always be implemented in such a way as to minimize unneeded effort for the faculty and maximize chances of actual improvement to teaching. This means that information should be requested when it is timely to reflect on the relevant process, and providing the information should be efficient and non-cumbersome. The information should also be the start of a dialogue, not a folder dropped into the memory hole. If needed to achieve these goals, policy should be implemented more slowly and/or less uniformly.

Finally, teaching faculty needs to the main actor in the story, not a cynical onlooker. The shift from input control to output control, combined with the highly competitive academic environment and the lack of job certainty for many (junior) teachers can easily lead to a feeling of inadequacy and underappreciation, and can lead to cynicism, lower performance, and even medical problems. Where quality monitoring is directly related to evaluation, the metrics should be fair, transparent, and predictable. Where quality metrics are not related to evaluation, they should be discussed constructively and with understanding for the efforts of the teacher and the specifics of the case. Effort by itself is not enough, but the effort should be appreciated even if the measured output is not as desired.

In sum, there are many opportunities for using educational assessment and quality monitoring instruments to improve the quality of education. If these instruments are mainly seen by faculty as “feeding the bureaucratic beast”, however, it is quite likely that they will not contribute to a real quality culture, but rather cause a loss of perceived professional autonomy and frustration over time spent “ticking boxes” rather than preparing classes or giving feedback to students. I have provided a number of recommendations to achieve more constructive monitoring. In short, goals should be clearly and sincerely explained; measurement should be close to the substantive goals; monitoring should be minimally obtrusive; and faculty needs to be included as the main actor rather than as the object of monitoring. Following these recommendations may sometimes lead to fewer boxes being ticked, but will hopefully contribute to a real improvement in teaching quality and job satisfaction and productivity of teaching faculty.

Sources

Biesta , G. J. J. (2010) Good Education in an Age of Measurement, Boulder: Paradigm Publishers.

Biesta, G. (2015). What is education for? On good education, teacher judgement, and educational professionalism. European Journal of Education, 50(1), 75-87.

Hoecht, A. (2006). Quality assurance in UK higher education: Issues of trust, control, professional autonomy and accountability. Higher education, 51(4), 541-563.

Newton, J. (2000). Feeding the Beast or Improving Quality?: academics’ perceptions of quality assurance and quality monitoring. Quality in higher education, 6(2), 153-163.

Power, M. (1997). The Audit Society. Oxford: Oxford University Press.

New journal launched: Computational Communication Research

I am very excited to announce that we* just launched Computational Communication Research (CCR), a new open-access peer-reviewed journal dedicated to development and applications of computational methods for communication science. We hope that CCR will serve as central home for communication scientists with an interest in and focus on computational methods — a place to read and publish the cutting edge work in our growing subfield.

Please see the inaugural call for papers at http://computationalcommunication.org/inaugural-cfp/ (abstracts 30 Sept, manuscripts 30 Nov), and consider submitting your best computational work to the first issue!

Don’t hesitate to email me for more information, and looking forward to your submissions!

Vacancy: PhD student with interest in mobile news and filter bubbles

[https://www.vu.nl/nl/werken-bij-de-vu/vacatures/2018/18161phdpositionincomputationalcommunicationscience.aspx]

The PhD Candidate will work as part of the Project Team “Inside the filter bubble: A framework for deep semantic analysis of mobile news consumption traces“, a collaboration between the VU, UvA (University of Amsterdam), CWI (Centre for Mathematics and Computer Science) and the Netherlands eScience Centre.

In this project we will develop and use tracking techniques to analyse mobile news consumption patterns to determine the impact of biased and homogeneous news diets on political knowledge and attitudes. Online and mobile news consumption leaves digital traces that are used to personalize news supply, possibly creating filter bubbles where people are exposed to a low diversity of issues and perspectives that match their preferences. Filter bubbles can be detrimental for the role of journalism in democracy and are therefore subject to considerable debate among academics and policymakers alike. The goal of this project is to develop techniques for analysing (mobile) news consumption patterns to determine the impact of selective news exposure on political knowledge and attitudes. This will contribute to the quality of (digital) journalism, news supply and democracy.

Tasks
Together with the other members of the project team, you will conduct the substantive analysis of these data and write papers as outlined in the project proposal, culminating in a dissertation to be defended at the VU. See ccs.amsterdam/projects/jeds for the project description or mail us for the full proposal.

Requirements
We are looking for a candidate that has either a degree in communication science, journalism, or related with strong affinity with computational methods; or a degree in data science, computer science or similar with strong affinity with news and societal problems. Candidates that expect to obtain their degree in the near future are also invited to apply.

[https://www.vu.nl/nl/werken-bij-de-vu/vacatures/2018/18161phdpositionincomputationalcommunicationscience.aspx]

 

 

Vacancies for asst. profs. in computational (political) communication

The VU University Amsterdam has three openings for assistant professors, of which two explicitly look for candidates with computational skills:

https://www.academictransfer.com/en/46787/assistant-professors-universitair-docenten/

The VU department of Communication Science aims to hire candidates in the fields of:

a) Political communication / public affairs (from January 2019)

b) Corporate communication and/or marketing communication (from January 2019),

c) Media psychology. (from August 1st)

For all three positions, we prefer candidates with a strong focus on the use of new communication technologies (e.g. social media, social robotics, sharing platforms). For positions a) and b) we prefer candidates who apply computational methods such as automatic text analysis, machine learning, or network analysis using programming languages such as R and Python.

If you have are an expert in any of these fields, have good computational skills, and want to be part of our fast-growing Computational Communication Science lab (see also http://ccs.amsterdam), please consider applying to one of these positions before May 10th.

Don’t hesitate to email me if you need more information! (wouter@vanatteveldt.com)