Greetings,
I'll be posting the online assignment later this week.
You'll have a full week to complete it once it's posted.
Jack
Tuesday, February 24, 2015
Tuesday, February 17, 2015
Sports Production Jobs/Internships
If you're interested in a career in sports production, don't miss this:
Representatives from Game Creek Video, live event mobile production, will visit the Park School Feb. 19 and 20 to recruit for their postgrad one-year apprentice program, and summer internships.
Meet with Game Creek in the first floor Park Photo Gallery #184 either Thurs. at 5:00 p.m. or Friday at 11:00 a.m. If neither time works, please contact Eloise Greene atEloise@ithaca.edu.
Game Creek clients include ESPN, Fox Sports, NBC Sports, The NFL Network, The YES Network and MSG Network.
Saturday, February 14, 2015
Class on Monday
Greetings,
As you may or may not recall, there was a chance that I would be at a conference this week (and that our class would be online).
As it turns out, however, I will not be at a conference. Instead, I need to be with my mom as she is facing some serious health issues.
This means that we won't be meeting for class on Monday in the regular fashion.
Instead, please use this time to 1) prepare for the exam, and 2) complete the SD, SE(p) and the z-score problems on Sakai (under tests and quizzes).
The exam will still be on Wednesday.
You may use any and all resources to complete it (the book, notes, the blog, the Internet, etc.). The only rule is that you must complete the exam on your own. That is, no one can help you.
You may take the exam anywhere you choose, provided you have an Internet connection.
It will be released at class time and you will have 2 hours to complete it.
If all goes well, I should be back to campus late Thursday, early Friday.
Jack
As you may or may not recall, there was a chance that I would be at a conference this week (and that our class would be online).
As it turns out, however, I will not be at a conference. Instead, I need to be with my mom as she is facing some serious health issues.
This means that we won't be meeting for class on Monday in the regular fashion.
Instead, please use this time to 1) prepare for the exam, and 2) complete the SD, SE(p) and the z-score problems on Sakai (under tests and quizzes).
The exam will still be on Wednesday.
You may use any and all resources to complete it (the book, notes, the blog, the Internet, etc.). The only rule is that you must complete the exam on your own. That is, no one can help you.
You may take the exam anywhere you choose, provided you have an Internet connection.
It will be released at class time and you will have 2 hours to complete it.
If all goes well, I should be back to campus late Thursday, early Friday.
Jack
Friday, February 13, 2015
SD, z-score, and SE(p) problems are posted on Sakai
SD, z-score, and SE(p) problems are posted on Sakai (under tests & quizzes).
Remember, the goal is to help you practice/prepare for the exam. You are allowed to complete the exercises until you get them right.
Jack
Remember, the goal is to help you practice/prepare for the exam. You are allowed to complete the exercises until you get them right.
Jack
Wednesday, February 11, 2015
$$$ Stress hurting Americans' health
February 4, 2015
American Psychological Association Survey Shows Money Stress Weighing on Americans’ Health Nationwide
Stress in America™ survey finds parents, younger generations and lower-income households have higher stress than others overall
WASHINGTON — While aspects of the U.S. economy have improved, money continues to be a top cause of stress for Americans, according to the new Stress in America™: Paying With Our Health survey released today by the American Psychological Association. According to the survey, parents, younger generations and those living in lower-income households report higher levels of stress than Americans overall, especially when it comes to stress about money.
“Regardless of the economic climate, money and finances have remained the top stressor since our survey began in 2007. Furthermore, this year’s survey shows that stress related to financial issues could have a significant impact on Americans’ health and well-being,” APA CEO and Executive Vice President Norman B. Anderson, PhD, said.
The survey, which was conducted by Harris Poll on behalf of APA among 3,068 adults in August 2014, found that 72 percent of Americans reported feeling stressed about money at least some of the time during the past month. Twenty-two percent said that they experienced extreme stress about money during the past month (an 8, 9 or 10 on a 10-point scale, where 1 is “little or no stress” and 10 is “a great deal of stress”). For the majority of Americans (64 percent), money is a somewhat or very significant source of stress, but especially for parents and younger adults (77 percent of parents, 75 percent of millennials [18 to 35 years old] and 76 percent of Gen Xers [36 to 49 years old]).
A gap also appears to be emerging in stress levels between people living in lower-income (making less than $50,000 per year) and higher-income households that mirrors the growing wealth gap nationwide. In 2007, there was no difference in reported average stress levels between those who earned more and those who earned less than $50,000, with both groups reporting the same average levels of stress (6.2 on a 10-point scale). By 2014, a clear gap had emerged with those living in lower-income households reporting higher overall stress levels than those living in higher-income households (5.2 vs. 4.7 on the 10-point scale).
Stress about money and finances appears to have a significant impact on many Americans’ lives. Some are putting their health care needs on hold because of financial concerns. Nearly 1 in 5 Americans say that they have either considered skipping (9 percent) or skipped (12 percent) going to the doctor when they needed health care because of financial concerns. Stress about money also impacts relationships: Almost a third of adults with partners (31 percent) report that money is a major source of conflict in their relationship.
The report also uncovered good news about stress management. Americans who say they have someone they can ask for emotional support, such as family and friends, report lower stress levels and better related outcomes than those without emotional support. Unfortunately, some Americans say that they do not have anyone to rely on for emotional support. According to the survey, 43 percent of those who say they have no emotional support report that their overall stress has increased in the past year, compared with 26 percent of those who say they have emotional support.
On average, Americans’ stress levels are trending downward: The average reported stress level is 4.9 on a 10-point scale, down from 6.2 in 2007. Regardless of lower stress levels, it appears that Americans are living with stress levels higher than what we believe to be healthy — 3.7 on a 10-point scale — and some (22 percent) say they are not doing enough to manage their stress.
“This year’s survey continues to reinforce the idea that we are living with a level of stress that we consider too high,” Anderson said. “Despite the good news that overall stress levels are down, it appears that the idea of living with stress higher than what we believe to be healthy and dealing with it in ineffective ways continues to be embedded in our culture. All Americans, and particularly those groups that are most affected by stress — which include women, younger adults and those with lower incomes — need to address this issue sooner than later in order to better their health and well-being.”
To read the full Stress in America report or download graphics, visit the webpage.
For additional information on stress, lifestyle and behaviors, visit the APA Help Center webpage and read APA’s Mind/Body Health campaign blog. Join the conversation about stress on Twitter by following @APAHelpCenter and #stressAPA.
Methodology
The Stress in America survey was conducted online within the United States by Harris Poll on behalf of the American Psychological Association between Aug. 4 and 29, 2014, among 3,068 adults ages 18 and older who reside in the U.S. Because the sample is based on those who were invited and agreed to participate in the Harris Poll online research panel, no estimates of theoretical sampling error can be calculated. To read the full methodology, including the weighting variables, visit the Stress in America Press Room webpage.
The American Psychological Association, in Washington, D.C., is the largest scientific and professional organization representing psychology in the United States. APA’s membership includes nearly 130,000 researchers, educators, clinicians, consultants and students. Through its divisions in 54 subfields of psychology and affiliations with 60 state, territorial and Canadian provincial associations, APA works to advance the creation, communication and application of psychological knowledge to benefit society and improve people’s lives.
How to solve SD in plain English
Suppose you're looking to find the SD of a sample of these numbers:
1,2,2,2,3,4,9,9
If your mind works better using words over formulas, you can solve for the SD by following these steps:
1. Solve for the average (mean) of the values reported. In this case, the average is 4.
2. Find the difference between each value reported and the mean.
1- 4 = -3
2- 4= -2
2- 4= -2
2-4= -2
3- 4= -1
4- 4= 0
9- 4= 5
9- 4= 5
3. Square each of the differences (as shown below).
-3*-3 = 9
-2*-2= 4
-2*-2 = 4
-2 * -2 = 4
-1*-1 = 1
0*0 = 0
5*5 = 25
5*5 = 25
4. Sum the totals:
9+4+4+4+1+0+25+25 = 72
5. Divide by N-1
72/7 = 10.286
6. Take the square root.
The square root of 28.57 = 3.207
The SD is 3.21.
1,2,2,2,3,4,9,9
If your mind works better using words over formulas, you can solve for the SD by following these steps:
1. Solve for the average (mean) of the values reported. In this case, the average is 4.
2. Find the difference between each value reported and the mean.
1- 4 = -3
2- 4= -2
2- 4= -2
2-4= -2
3- 4= -1
4- 4= 0
9- 4= 5
9- 4= 5
3. Square each of the differences (as shown below).
-3*-3 = 9
-2*-2= 4
-2*-2 = 4
-2 * -2 = 4
-1*-1 = 1
0*0 = 0
5*5 = 25
5*5 = 25
4. Sum the totals:
9+4+4+4+1+0+25+25 = 72
5. Divide by N-1
72/7 = 10.286
6. Take the square root.
The square root of 28.57 = 3.207
The SD is 3.21.
Standard Deviation Overview
Standard Deviation and Music Tests
My question relates to music tests. A while ago, you mentioned that in addition to the average score and a few other things, that it’s also important to check the standard deviation for each song to see how much the respondents agreed in their evaluation of the songs. Can you explain that in more detail?
I’ll be happy to explain it.
If you simply look at a song’s mean (average) score, you don’t know how that scored was achieved. That is, let’s say that on a 10-point rating scale, a song receives a score of 5.5—right in the middle of the scale.
The first reaction to such a score is probably something like, “That’s an average testing song; not a hit; and not a piece of trash. But is that true? The standard deviation tells you the amount of agreement among the respondents in reference to their song scores. (I know you could look at the raw scores to see how they are distributed, but if you have 100 respondents and test 400+ songs, that’s a lot of data to review. You’d go nuts looking at all those numbers. The standard deviation speeds up the process.)
To make this easier to see, I developed a sample spreadsheet for three songs—each with an average (mean) score of 5.5. While each song’s average is 5.5, the average was arrived at in three unique ways. Look at the bottom of the table to see the standard deviations.
| Respondent | Song 1 | Song 2 | Song 3 |
| 1 | 1 | 1 | 5 |
| 2 | 1 | 3 | 5 |
| 3 | 1 | 3 | 5 |
| 4 | 1 | 3 | 5 |
| 5 | 1 | 4 | 5 |
| 6 | 1 | 4 | 5 |
| 7 | 1 | 4 | 5 |
| 8 | 1 | 5 | 5 |
| 9 | 1 | 5 | 5 |
| 10 | 1 | 6 | 5 |
| 11 | 10 | 6 | 6 |
| 12 | 10 | 6 | 6 |
| 13 | 10 | 6 | 6 |
| 14 | 10 | 7 | 6 |
| 15 | 10 | 7 | 6 |
| 16 | 10 | 7 | 6 |
| 17 | 10 | 8 | 6 |
| 18 | 10 | 8 | 6 |
| 19 | 10 | 8 | 6 |
| 20 | 10 | 9 | 6 |
Mean
| 5.5 | 5.5 | 5.5 |
Standard Deviation
| 4.6 | 2.1 | 0.5 |
Song 1 has a standard deviation of 4.6, which is high for a 10-point scale and shows that there isn’t a lot of agreement among the respondents—the song is highly polarized—respondents either hate it or love it. Song 2 has a standard deviation of 2.1—more agreement than Song 1, but still shows that the song is not universally liked or disliked (there is some polarization). Song 3 has a standard deviation of 0.5, indicating that there is a lot of agreement among the respondents.
Get the idea here? In addition to the average score, you need to look at the standard deviation (standard/average difference from the mean) of the songs to determine how much the respondents agree in their ratings. A song that receives a standard deviation of 0.0 indicates that everyone rated the song the same way. The higher the standard deviation, the more disagreement there is among the respondents. (The standard deviation can’t be higher than the highest number in the rating scale.)
Z-scores: The most versatile statistic for media research
z-scores
While there are dozens of statistical methods available to analyze data, the simple z-score is probably the most versatile. The following information is from Roger Wimmer's The Research Doctor Archive:
What are z-scores?
What are z-scores?
Whenever we conduct a test of any kind (e.g., tests in school, music tests, personality ratings, or even Arbitron ratings), we collect some type of scores. The next logical step is to give meaning to these numbers by comparing them to one another. For example, how does a vocabulary test score of 95 compare to a score 84? In a music test, how does a song score of 82 compare to a score of 72? And so on. Without these comparisons, the scores have no meaning.
Although there are different ways to compare scores to one another (e.g., percentile ranks), the best way is to determine each score's standard deviation (or "average difference") above or below the mean of the total group of scores. This placement in standard deviation units above or below the mean is called a z-score, or standard score.
But wait. What is standard deviation? To understand that, you need to understand another term—variance. In simple terms, variance indicates the amount of difference that exists in a set of scores or data—how much the data vary. If the variance, which differs from one test to another, is large, it means that the respondents did not agree very much on whatever it was they were rating or scoring. Obviously, if the variance is small, the respondents agreed (were similar) in their ratings or scores.
The standard deviation is the square root of the variance. The advantage of the standard deviation is that it is expressed in the same units as the original scores. For example, if you test something using a 10-point scale, the standard deviation will exist somewhere between 1 and 10; a 65-point test will have a standard deviation between 1 and 65. And so on.
The standard deviation (SD) is important because it is used in calculating z-scores. What we need are symbols for Score (X), Mean (M), and standard deviation (SD). With those symbols, the z-score formula is X - M / SD, or subtract the Mean of the group of scores from each individual score, and divide by the standard deviation. The typical way the z-score formula is shown is:
Z =
|
X- M
|
SD
|
All z-scores have a mean of zero and standard deviation of 1, and ranges (roughly) between –3.00 and +3.00. A z-score of "0" is average; a positive z-score is above average; a negative z-score is below average. Here is a picture of a normal curve, showing that about 68% of a sample falls between -1 and +1 standard deviations, and about 98% fall between -2 and +2 standard deviations.
z-scores relate to the normal curve (the bell curve you may remember from when your teacher said that he/she was going to "curve" the test scores). Because of this, we know where things "stand" when using z-scores. For example, about 68% off your scores for a test will fall between –1.00 and +1.00 z-scores (standard deviations) from the mean.
z-scores allow you to compare "apples to oranges." For example, when you conduct a music test, you can't compare the raw scores of the males to the raw scores of the females, or one age cell to another. But you can withz-scores. In addition, if you compute z-scores for a music test, you can compare the scores from one market to scores in another market. You can't do this with the raw scores (regardless of the rating scale you use).
By the way, z-scores are known as standard scores because they are computed by transforming your scores into another metric by using the standard deviation. Get it? standard-ized scores. This procedure is known as a monotonic transformation . . . that is, all of your scores were transformed to another form using the same (monotonic) z-score formula.
Sampling Error- SE(p) Overview
Sampling Error
Research error is not an easy topic to discuss because it involves a variety of concepts, words, and formulas. Because so many different types of people read this column, I have tried to simplify this discussion so just about anyone can understand it.
However, with that said, you may want to get a 6-pack of your favorite beverage because this answer is a bit long. OK, so on to your questions . . .
All research that analyzes a sample selected from a population has some degree of sampling error or standard error, both of which are abbreviated "SE." Sampling error is the estimated difference between the data from a sample and the "true" or "real" value that would be found if a census (analyzing the entire population) were conducted. There is no sampling error when a census is conducted because the calculations are based on the entire population, but a census still contains measurement error and random error, so even a census is not a perfect measurement.
Now, before I send you to the Arbitron sampling error tables, I need to explain a few basics about sampling error. First, there are two terms or concepts that important in sampling error: Confidence Interval andConfidence Level.
Confidence Interval is the "±" percentage you hear about when a research study is conducted. For example, a TV news anchor may report the results of a political poll and say something like, "The results of the poll have a margin of error of plus or minus 5 points (or 5 percent). The plus or minus 5 percent is the Confidence Interval for the study.
Confidence Level is also a percentage and provides an indication of how confident a person can be with the results of a study. For example, the 95% Confidence Level indicates that a person can be 95% certain that the results fall within the range of the Confidence Interval; the 99% Confidence Level indicates that a person can be 99% certain that a study's results fall within the range of the Confidence Interval. Most behavioral researchers use the 95% confidence level; researchers in "hard" sciences of medicine, physics, and chemistry often use Confidence Intervals as high as 99.99% (or .0001).
Another way to understand the Confidence Level is that it provides an indication of how likely it is that the results of a study are due to error or chance. For example, the 95% Confidence Level means that there is a 5% probability that the results of the study are due to error, chance, or both. In other words, the higher the Confidence Level, the lower the probability that the results are spurious (false); the lower the Confidence Level, the more likely that the results are due to error or chance. (Most behavioral science researchers use the 95% or 99% Confidence Level, although the 90% Level is occasionally be used in exploratory research—studies where the investigator is in the preliminary stages of an investigation.
There is no rule related to the Confidence Level used in a research project. Literally any Confidence Level can be used, and the decision to use one Confidence Level over another is purely arbitrary. The most fundamental reason for selecting one Confidence Level is the desire for accuracy (remember . . . the higher the Confidence Level, the less likely the probability that results are due to error or chance).
For example, let's say that a radio researcher uses the 90% Confidence Level. This means that the researcher (or client) is willing to accept the fact that there is a 10% probability that the results are due to error or chance. There is nothing wrong with this as long as everyone involved in the research knows and understands the "rules" of the study. However, researchers in the hard sciences would never use such a low Confidence Level. As mentioned, most researchers in the hard sciences often use Confidence Levels of 99.99%, which means that there is only one chance in 1,000 that the results are due to error or chance. Can you imagine the criticism a cancer researcher would receive if the results from a study testing for new drug were tested at the 90% Confidence Level? Gag me.
I hope I haven't lost anyone yet, because this is important stuff if you want to read, interpret, and understand any research, including Arbitron and Nielsen ratings.
Summary up to this point . . . When you read any research study, you need to know the Confidence Interval and the Confidence Level. The report, or the researcher, should say something like, "The results of this study have a Confidence Interval of ±5% at the 95% Confidence Level." If you don't know these two pieces of information, you don't know how accurate the results are . . . ya gots ta know.
OK, Now we need to look at the formula for sampling error (remember, this is the Confidence Interval). The formula for the Confidence Interval at the 95% Confidence Level:
where p is the result from the study, N = sample size, and Z represents the area under the normal curve related to the desired Confidence Level.
So, if 20% of the respondents from a sample of 500 give the same answer, the formula for the 95% confidence level (discussed below) is:
The 20% response has a maximum estimated sampling error of ±3.5%, which means that the "actual" or "real" percentage for the answer is somewhere between 16.5% and 23.5%.
Now, if you wanted to interpret the results at the 90% Confidence Level, you would multiply the result under the square root symbol times 1.64; at the 99% Confidence Level, multiply times 2.57.
Note: The corresponding Z-scores in the formula come from a table known as the "Areas Under the Normal Curve." If you want to see the table, I prepared one our textbook, Mass Media Research: An Introduction—just click here. By the way, the Z-score for the 68% Confidence Level is 1.0, so there is no need to multiply the result under the square root.
So why is the discussion about Confidence Levels important? Because the sampling error tables supplied by Arbitron are calculated at the 68% Confidence Level, which means that you can only be 68% certain that the data you interpret fall within the range of your Confidence Interval. In other words, there is a 32% chance that the results you are reading are due to error or chance.
One more thing . . . The reason I explained the sampling error formula is that it is discussed on Arbitron's sampling error pages, but the formula is in prose, not numbers and symbols. You need to know the origin of the sampling error.
So . . . Arbitron does have sampling error tables on the Internet, and you can view them by clicking here.
That's the answer to your first question. On to the second . . .
You also ask what sampling error would I consider "OK," and what would cause me concern. Well, we're dealing with behavioral research here, and humans are very difficult to describe and their behavior is particularly difficult to predict. I don't like to exceed a Confidence Interval of 5%...
Extra credit opportunity
If you attend the event below, you will receive +3 points added to your first exam score:
Active Minds at Ithaca College is hosting "Change the Conversation," a presentation and discussion event that examines the ways that the mass media portray issues of mental health and illness. We're interested in helping students become more aware and responsible consumers and creators of media content, and believe that the event applies particularly well to students in the TV-R department.
"Active Minds Presents... Change the Conversation" takes place on February 17 (Tuesday) from 7pm to 8pm in Park 279. Students can (but are not required to) indicate their plan to attend at https://www.facebook.com/
Review Sheet #1 MMRM
Review Sheet MMRM Exam #1
Spring 2015
Science and Research
Definition of research
Characteristics of scientific method (public, objective, empirical, systematic, cumulative, predictive)
Typical steps of the research process
Validity and Reliability
Importance of replication
Variables (intervening, confounding)
Theory
Hypotheses
Different ways of knowing
Elements of Research
Concept/construct
IV and DV
Four levels of measurement (nominal, ordinal, interval, ratio)
Major scales (Likert, semantic differential, simple)
Major types of validity (face, predictive, concurrent, construct)
Some major threats to validity
Hypotheses vs. Research Questions
Methodologies: Experiments, Surveys, Content Analysis
Sampling
Census vs. Sample
Probability vs. nonprobability
Random, systematic random, convenience samples
Sampling Error (Margin of Error)
How to compute Sampling Error
Confidence Level vs. Confidence Interval
What is the appropriate CL for most social science research (as a %)?
Appropriate z-score value
Ethics
Three major theoretical approaches (Deontological, Teleological, Relativistic)
Four major principles (autonomy, nonmaleficence, beneficence, justice)
Four special concerns of social science research: voluntary participation, informed consent, concealment, deception
Practice Problems
Practice Problems
Here is a practice SE problem (margin of error):
Nielsen is interested in seeing if the Northeast has different viewing patterns than the rest of the country, so they systematically choose a random sample of 900 homes and measure the TV ratings. They note that the rating for Sunday Night Football is 21. What is the SE(p)? What is the confidence interval and what is the appropriate confidence level for social science?
Here is a sample SD problem:
An executive at SyFy wants to know which new original movie to air this spring. Her choices are between two made-for-tv movies-- "SharkZombie" or "VampNado."
The first show was tested across the country and returned acceptance scores of
50
50
50
40
20
30
(on a scale of 1-60), and the second show returned scores of
50
40
40
40
30
40
Which show should she choose and why?
Here is a sample z-Score problem:
Which show had the better year, American Idol or X-Factor? How do you know?
Average rating of American Idol: 6.9
SD: 2.1
This year's rating of American Idol: 7.2
Average rating of The Voice: 7.5
SD: 2.0
This year's rating of The Voice: 7.8
Sunday, February 8, 2015
Steps to the Scientific Method
1. Select a problem.
2. Review existing research and theory (when relevant).
3. Develop hypotheses or research questions.
4. Determine an appropriate methodology/research design.
5. Collect relevant data.
6. Analyze and interpret the results.
7. Present the results in an appropriate form.
8. Replicate the study (when necessary).
2. Review existing research and theory (when relevant).
3. Develop hypotheses or research questions.
4. Determine an appropriate methodology/research design.
5. Collect relevant data.
6. Analyze and interpret the results.
7. Present the results in an appropriate form.
8. Replicate the study (when necessary).
Sample Problems
Here is a practice SE problem (margin of error):
Nielsen is interested in seeing if the Northeast has different viewing patterns than the rest of the country, so they systematically choose a random sample of 800 homes and measure the TV ratings. They note that the rating for Sunday Night Football is 25. What is the SE?
Here is a sample SD problem:
An executive at ABC wants to know which sitcom to air this fall. Her choices are between two shows-- "Welcome home, son!" or "Married with Children... Again."
The first show was tested across the country and returned acceptance scores of 5,5,5,4,1 (on a scale of 1-5), and the second show returned scores of 5,4,4,4,3.
Which show should she choose and why?
Here is a sample z-Score problem:
Which show had the better year relative to expectations, The Bachelor or The Bachelorette? How do you know?
Average rating of The Bachelor: 6.1
SD: 4
This year's rating of The Bachelor: 7.1
Average rating of The Bachelorette: 4.5
SD: 2
This year's rating of The Bachelorette: 5.3
Sampling Overview
Some scientific research examines every member of a population, a process called a census.
However, in most situations, it is impractical to examine every member of a population. In these instances, researchers draw a sample from the population. A sample is defined as “a subset of the population that is representative of the entire population.”
There are two types of samples: probability samples and nonprobability samples.
A probability sample follows mathematical guidelines and allows researcher to calculate the estimated sampling error present in a study.
Probability samples include random samples, where each subjects in a population has an equal chance of being selected; systematic random samples, stratified samples, and cluster samples.
In addition, there are several types of nonprobability samples, including available samples, volunteer samples, purposive samples, and quota samples.
Sampling methods must be selected carefully by considering issues such as the purpose of the study, cost versus value, time constraints, and the amount of acceptable sampling error.
Think of probability (or representational) sampling along the following lines--
Suppose you were going to make a giant pot of chicken noodle soup-- 10 gallons. You throw in egg noodles, chicken, broth (chicken stock), potatoes, onions, celery, salt, pepper, carrots, etc. You cook the soup for hours and now it's time to get a sense of how it tastes.
If you were to consume the entire 10 gallons of soup before you declare how it tastes, we would call that a census. You could have, of course, just taken a ladle and tasted that. There is a catch, though-- the soup must be mixed up really well so that your ladle is a fair representation of the entire 10-gallon pot.
If you dip your ladle in and pull out a fair representation of noodles, chicken, broth, potatoes, onions, celery, salt, pepper, etc., then we conclude that the rest of the soup tastes the same.
Now let's assume that we accidentally scrape our ladle against all the built up gunk along the side of the pot before we pull it up. In this case, we taste our ladle full of soup and we are NOT impressed. It tastes disgusting and, in fairness, is not really a fair representation of what the soup tastes like. If, however, we don't take a different ladle full, we'll never know that. In this instance, our ladle is NOT a fair representation of the entire 10-gallon pot.
Well, this is what we do in science, too.
We don't need to measure all 310 million Americans to get a sense, for example, of what they watched on TV last night (trust me-- American Idol was the #1 show). Instead, as long as we take a ladle full of Americans (assuming that the pot-- in this case, the country-- is all mixed up), we should be able to get a taste of the entire population.
This is precisely what the Nielsen Media Research Company does when it reports the overnight TV ratings. Rather than asking ALL Americans what they watched, they ask only a small, representational sample and that's enough to get a sense as to what the rest of the country is doing, too.
How small? The TV ratings (representing what 110 million households watch) are generally garnered by asking a few thousand homes.
You may be thinking-- "There is NO WAY that a few thousand homes can adequately represent what 110 million households watched last night!"
Well, the data would suggest otherwise. It is quite accurate.
Let's look at the previous presidential election for further evidence.
A few days before the 2008 election, CNN surveyed voters across the country. Out of a country of 310 million people, they asked ONLY 714 potential voters who they planned on voting for in the presidential election.
Here were their reported results:
Sample size: 714
Obama-- 53%
McCain--46%
Meanwhile, in a DIFFERENT survey asking DIFFERENT people (but still representative), the McClatchy Group (a company that owns several newspapers) conducted its own poll and asked 760 likely voters who they planned on voting for and here were their reported results:
Sample size: 760
Obama--53%
McCain--46%
As for the REAL results from the election? Here they are--
Obama-- 52.9%
McCain--45.6%
How many people voted?
Just under 123 million...
Amazing, isn't it?
CNN was able to ask just over 700 people, yet they got the SAME results as they would have had they asked 123 MILLION people.
That, my friends, is the beauty of sampling.
It is science's best friend (social sciences and the natural sciences).
Is it fool-proof? Nope. That's why each legitimate survey sample will report a Margin of Error and a Confidence Level.
In social science, most confidence levels are 95%. Furthermore, we tend to use 3.5% as an acceptable margin of error.
In the CNN case, the margin of error is 3.5%. This means that CNN was 95% confident that the ACTUAL results would be +/- 3.5%. This is to say that they were confident that they actual results would be something like this--
Obama-- 49.5% - 56.5%
McCain-- 42.5% - 49.5%
As it turns out, their REAL margin of error was 0.1 for Obama and 0.4 for McCain.
Ok, now for our final project (which you'll work on alone, but I'll help with each step), we'll use a random sample of magazine covers (that I'll provide). We'll follow the same logic that CNN did. Rather than look at thousands of covers, we'll only look at dozens (but, if we do our job well, it will be representational of the overall population).
Get it?
Jack
However, in most situations, it is impractical to examine every member of a population. In these instances, researchers draw a sample from the population. A sample is defined as “a subset of the population that is representative of the entire population.”
There are two types of samples: probability samples and nonprobability samples.
A probability sample follows mathematical guidelines and allows researcher to calculate the estimated sampling error present in a study.
Probability samples include random samples, where each subjects in a population has an equal chance of being selected; systematic random samples, stratified samples, and cluster samples.
In addition, there are several types of nonprobability samples, including available samples, volunteer samples, purposive samples, and quota samples.
Sampling methods must be selected carefully by considering issues such as the purpose of the study, cost versus value, time constraints, and the amount of acceptable sampling error.
Think of probability (or representational) sampling along the following lines--
Suppose you were going to make a giant pot of chicken noodle soup-- 10 gallons. You throw in egg noodles, chicken, broth (chicken stock), potatoes, onions, celery, salt, pepper, carrots, etc. You cook the soup for hours and now it's time to get a sense of how it tastes.
If you were to consume the entire 10 gallons of soup before you declare how it tastes, we would call that a census. You could have, of course, just taken a ladle and tasted that. There is a catch, though-- the soup must be mixed up really well so that your ladle is a fair representation of the entire 10-gallon pot.
If you dip your ladle in and pull out a fair representation of noodles, chicken, broth, potatoes, onions, celery, salt, pepper, etc., then we conclude that the rest of the soup tastes the same.
Now let's assume that we accidentally scrape our ladle against all the built up gunk along the side of the pot before we pull it up. In this case, we taste our ladle full of soup and we are NOT impressed. It tastes disgusting and, in fairness, is not really a fair representation of what the soup tastes like. If, however, we don't take a different ladle full, we'll never know that. In this instance, our ladle is NOT a fair representation of the entire 10-gallon pot.
Well, this is what we do in science, too.
We don't need to measure all 310 million Americans to get a sense, for example, of what they watched on TV last night (trust me-- American Idol was the #1 show). Instead, as long as we take a ladle full of Americans (assuming that the pot-- in this case, the country-- is all mixed up), we should be able to get a taste of the entire population.
This is precisely what the Nielsen Media Research Company does when it reports the overnight TV ratings. Rather than asking ALL Americans what they watched, they ask only a small, representational sample and that's enough to get a sense as to what the rest of the country is doing, too.
How small? The TV ratings (representing what 110 million households watch) are generally garnered by asking a few thousand homes.
You may be thinking-- "There is NO WAY that a few thousand homes can adequately represent what 110 million households watched last night!"
Well, the data would suggest otherwise. It is quite accurate.
Let's look at the previous presidential election for further evidence.
A few days before the 2008 election, CNN surveyed voters across the country. Out of a country of 310 million people, they asked ONLY 714 potential voters who they planned on voting for in the presidential election.
Here were their reported results:
Sample size: 714
Obama-- 53%
McCain--46%
Meanwhile, in a DIFFERENT survey asking DIFFERENT people (but still representative), the McClatchy Group (a company that owns several newspapers) conducted its own poll and asked 760 likely voters who they planned on voting for and here were their reported results:
Sample size: 760
Obama--53%
McCain--46%
As for the REAL results from the election? Here they are--
Obama-- 52.9%
McCain--45.6%
How many people voted?
Just under 123 million...
Amazing, isn't it?
CNN was able to ask just over 700 people, yet they got the SAME results as they would have had they asked 123 MILLION people.
That, my friends, is the beauty of sampling.
It is science's best friend (social sciences and the natural sciences).
Is it fool-proof? Nope. That's why each legitimate survey sample will report a Margin of Error and a Confidence Level.
In social science, most confidence levels are 95%. Furthermore, we tend to use 3.5% as an acceptable margin of error.
In the CNN case, the margin of error is 3.5%. This means that CNN was 95% confident that the ACTUAL results would be +/- 3.5%. This is to say that they were confident that they actual results would be something like this--
Obama-- 49.5% - 56.5%
McCain-- 42.5% - 49.5%
As it turns out, their REAL margin of error was 0.1 for Obama and 0.4 for McCain.
Ok, now for our final project (which you'll work on alone, but I'll help with each step), we'll use a random sample of magazine covers (that I'll provide). We'll follow the same logic that CNN did. Rather than look at thousands of covers, we'll only look at dozens (but, if we do our job well, it will be representational of the overall population).
Get it?
Jack
Research Ethics Overview
Research Ethics Overview
The ethical problems of doing scientific research should not be overlooked. Considering ethical problems is vital for doing any research project. There are many important ethical concerns in the research process, including why research must be ethical, general ethical theories, ethical principles, specific ethical problems, and ethics in data analysis and reporting.
Since most mass media research involves human beings, researchers must not violate the rights of participants. Ethical research is the right thing to do.
Three general types of theories are discussed that have evolved concerning ethics:
1) rule-based or deontological theories (based off categorical imperatives – principles that define appropriate action in any and all situations; Golden Rule: moral duty),
2) balancing or teleological theories (based on notion of utilitarianism; the good that may come from an action is weighed against or balanced against possible harm. The end may justify the means). , and
3) relativistic theories (no 'absolute' right or wrong way of behaving; ethical decisions are determined by cultural norms; everything is relative).
There are four relevant ethical principles:
1) autonomy or self-determination,
2) nonmaleficence,
3) beneficence, and
4) justice.
Autonomy suggests that researchers should respect the rights, values, and decisions of other people. As a way to guarantee this principle, researchers in mass media use informed consent.
Nonmaleficence means that it is wrong to intentionally inflict harm on others, while beneficence stipulates that a researcher should remove any existing harm and provide benefits to others.
Justice is related to the equal rights of participants, suggesting that participants should be treated equally and all benefits should be shared with all who are qualified.
Ethical issues to be concerned with when it comes to research include:
1) voluntary participation,
2) informed consent,
3) concealment,
4) deception, and
5) protection of privacy.
Research participation should be a voluntary process and informed consent provides information to the participants to aid them in making a choice. The researcher should warn of any possible discomfort or unpleasantness in the research process and obtain the consent form from the participants.
Concealment involves the withholding of certain information from the participants. Deception is intentionally providing false information. Although there are arguments concerning the pros and cons of both practices, these two techniques should not be used indiscriminately.
Researchers can use two ways to protect the privacy of participants: a promise of anonymity or confidentiality. Researchers are responsible for having a moral and ethical obligation in data analysis and reporting. Questionnaire responses and experimental observations should not be fabricated, altered, or discarded.
Online research raises special ethical problems. Passive analysis of online content generally raises fewer ethical issues than does active research where the investigator tries to gather information directly from online users.
The ethical problems of doing scientific research should not be overlooked. Considering ethical problems is vital for doing any research project. There are many important ethical concerns in the research process, including why research must be ethical, general ethical theories, ethical principles, specific ethical problems, and ethics in data analysis and reporting.
Since most mass media research involves human beings, researchers must not violate the rights of participants. Ethical research is the right thing to do.
Three general types of theories are discussed that have evolved concerning ethics:
1) rule-based or deontological theories (based off categorical imperatives – principles that define appropriate action in any and all situations; Golden Rule: moral duty),
2) balancing or teleological theories (based on notion of utilitarianism; the good that may come from an action is weighed against or balanced against possible harm. The end may justify the means). , and
3) relativistic theories (no 'absolute' right or wrong way of behaving; ethical decisions are determined by cultural norms; everything is relative).
There are four relevant ethical principles:
1) autonomy or self-determination,
2) nonmaleficence,
3) beneficence, and
4) justice.
Autonomy suggests that researchers should respect the rights, values, and decisions of other people. As a way to guarantee this principle, researchers in mass media use informed consent.
Nonmaleficence means that it is wrong to intentionally inflict harm on others, while beneficence stipulates that a researcher should remove any existing harm and provide benefits to others.
Justice is related to the equal rights of participants, suggesting that participants should be treated equally and all benefits should be shared with all who are qualified.
Ethical issues to be concerned with when it comes to research include:
1) voluntary participation,
2) informed consent,
3) concealment,
4) deception, and
5) protection of privacy.
Research participation should be a voluntary process and informed consent provides information to the participants to aid them in making a choice. The researcher should warn of any possible discomfort or unpleasantness in the research process and obtain the consent form from the participants.
Concealment involves the withholding of certain information from the participants. Deception is intentionally providing false information. Although there are arguments concerning the pros and cons of both practices, these two techniques should not be used indiscriminately.
Researchers can use two ways to protect the privacy of participants: a promise of anonymity or confidentiality. Researchers are responsible for having a moral and ethical obligation in data analysis and reporting. Questionnaire responses and experimental observations should not be fabricated, altered, or discarded.
Online research raises special ethical problems. Passive analysis of online content generally raises fewer ethical issues than does active research where the investigator tries to gather information directly from online users.
Elements of Research
Some of the important elements of research include --
concept,
construct,
variables,
measurement,
scales,
reliability, and
validity.
To conduct effective research, a researcher needs to have a clear understanding of these elements.
A concept is a term that expresses an abstract idea formed by generalizing from particulars and summarizing related observations. Researchers can simplify research by using concepts that helps them formulate a general and inclusive term.
A construct is a combination of concepts. Variables are used to describe the phenomena and events that can be measured in empirical world. Independent variables are varied by the researcher, whereas dependent variables are the ones that researcher wants to find out about. Researchers can observe the phenomena or events by a clear statement of what is to be observed, called an operational definition.
Measurement is an assignment of numerals to persons, objects, or characteristics. In this chapter, four levels of measurement are described. The nominal level simply assigns numerals to the objects without mathematical significance. The ordinal level ranks objects according to certain orders, such as from smallest to largest. The scale is at the interval level when the intervals between adjacent points are equal. The ratio level, the highest level of measurement, has all the properties of interval scales and plus a true zero point.
Measurement of some variables requires scales. This chapter describes Thurstone scales, Guttman scales, Likert scales, and semantic differential scales. Likert scales and semantic differential scales are the most commonly used scales in mass media research.
A measurement must be both reliable and valid to be useful in any research procedures. We can say a measure is reliable if it consistently gives the same answer. Reliability consists of three components: stability, internal consistency, and equivalency. To assess the reliability of measurements, researcher can use the test-retest method with the correlation coefficient. The split-half technique and the cross-test reliability method can be used to examine the internal consistency and the equivalency component of reliability. Also, intercoder reliability is used in the case of content analysis.
A valid measure measures what it is supposed to measure. Four major types of validity are: face validity (does it measure what it says it measures), predictive validity, (if SAT scores are valid indicators of college success, then those who do well on the exam should do better in college than those who don't) concurrent validity (can we compare our results to other readily available results concurrently to make sure things are valid), and construct validity (are our constructs valid?).
Reliability and validity are related. Reliability is a necessary condition to establish validity, but it is not a sufficient condition. A measurement can be reliable even if it is not valid. It is important to remember that a measurement must be both reliable and valid to be used in the research.
Ways of Knowing
There are a variety of ways to investigate a research question or hypothesis. In 1986, Kerlinger, using definitions provided nearly a century ago by C. S. Peirce, presents four approaches to finding answers, or “methods of knowing.” They are tenacity, intuition, authority, and science. As Wimmer & Dominick (2003) state:
The scientific method approaches learning as a series of small steps. That is, one study or one source provides only an indication of what may or may not be true; the “truth” is found through a series of objective analyses. This means that the scientific method is self-correcting in that changes in thought or theory are appropriate when errors in previous research are uncovered.
For example, in 1984 Barry Marshall, a medical resident in Perth, Australia, identified a bacterium (Helicobacter pylori or H. pylori) as the cause of stomach ulcers (not an increase in stomach acid due to stress or anxiety). After several years, hundreds of independent studies proved that Marshall was correct, and in 1996, the Food and Drug Administration (FDA) approved a combination of drugs to fight ulcers-an antacid and an antibiotic.
In this class, we adopt the scientific method way of learning.
A user of the method of tenacity follows the logic that something is true because it has always been true. An example is the storeowner, who says, “I don’t advertise because my parents did not believe in advertising.” The idea is that nothing changes-what was good, bad, or successful before will continue to be so in the future.
In the method of intuition, or the a priori approach, a person assumes that something is true because it is “self-evident” or “stands to reason.” Some creative people in advertising agencies resist efforts to test their advertising methods because they believe they know what will attract customers. To these people, scientific research is a waste of time.
The method of authority promotes a belief in something because a trusted source, such as a parent, a news correspondent, or a teacher, says it is true. The emphasis is on the source, not on the methods the source may have used to gain the information.
For example, in 1984 Barry Marshall, a medical resident in Perth, Australia, identified a bacterium (Helicobacter pylori or H. pylori) as the cause of stomach ulcers (not an increase in stomach acid due to stress or anxiety). After several years, hundreds of independent studies proved that Marshall was correct, and in 1996, the Food and Drug Administration (FDA) approved a combination of drugs to fight ulcers-an antacid and an antibiotic.
In this class, we adopt the scientific method way of learning.
Levels of Measurement
In scientific research, we examine the relationships between variables. Each of these variables, in quantitative research, can be measured. Specifically, there are levels of measurement--
1) Nominal-- A nominal (sometimes called categorical) variable is one that is typically measured in categories. It is the weakest form of measurement, but sometimes the only one available. For example, if we were to identify a person as being either "male" or "female," we don't have a lot of wiggle room-- you tend to be one or the other (biologically speaking and, yes, I'm fully aware that some people are born with sex organs of both males and females). Historically, many demographic variables are nominal. These include variables such as gender, race, religion, etc.
2) Ordinal-- An ordinal variable is one where a numeric value is assigned to the variables, but we can't assume an equal distance between the points. For example, if I ask you to list your top three favorite ice cream flavors, you might say that mint chocolate chip ranks first, french vanilla ranks second, and double-dutch chocolate ranks third. Well, based on this ranking, I know that mint chocolate chip is your favorite, but I can't tell how much MORE you like it compared to the others. You may like it 10x more than your second favorite, or it may be almost a tie between the two. I have no way of knowing. Ordinal variables indicate rank, but not distance between points.
Another way of thinking of it is to look at the results of a horse race. Sometimes a horse finishes first by 20 lengths, sometimes only by a nose. In either case, the horse still wins, but, statistically speaking, the distance between points matters.
3) Interval-- An interval level of measurement provides a rank order AND equal distance between the points, but has no real "zero." The Fahrenheit scale of temperature is an every day example. The difference between 70 and 80 is the same as the difference between 20 and 30 (10 degrees). The zero on the scale, however, is not a TRUE zero, since the measurement does not indicate a lack of the concept, but, rather, just one point on the continuum.
4) Ratio-- A ratio level of measurement provides rank order, equal distance AND has a true zero-- which indicates none of the concept. For example, if I'm measuring "TV viewing" in hours for the past day and I observe that you did not watch even one moment of TV during that time, then I can record "0."
Here are some sample variables. Indicate, based on how the variables is being measured, the level of measurement:
1. What is your gender?
1. Female
2. Male
3. Other: __________
2. What color hair does the cartoon character have?
1. Blonde
2. Brown
3. Black
4. Red
5. Other: ____________
3. Indicate your level of agreement with the following statement:
College is too expensive.
5-Strongly Agree 4- Agree 3- Neutral 2-Disagree 1-Strongly Disagree
4. Please indicate total your income for the year 2009: _________________
5. Please indicate your favorite electronic media from favorite (5) to least favorite (1):
a. Internet
b. Television
c. Music
d. Video games
e. Movies
Post your answers here.
Jack
1) Nominal-- A nominal (sometimes called categorical) variable is one that is typically measured in categories. It is the weakest form of measurement, but sometimes the only one available. For example, if we were to identify a person as being either "male" or "female," we don't have a lot of wiggle room-- you tend to be one or the other (biologically speaking and, yes, I'm fully aware that some people are born with sex organs of both males and females). Historically, many demographic variables are nominal. These include variables such as gender, race, religion, etc.
2) Ordinal-- An ordinal variable is one where a numeric value is assigned to the variables, but we can't assume an equal distance between the points. For example, if I ask you to list your top three favorite ice cream flavors, you might say that mint chocolate chip ranks first, french vanilla ranks second, and double-dutch chocolate ranks third. Well, based on this ranking, I know that mint chocolate chip is your favorite, but I can't tell how much MORE you like it compared to the others. You may like it 10x more than your second favorite, or it may be almost a tie between the two. I have no way of knowing. Ordinal variables indicate rank, but not distance between points.
Another way of thinking of it is to look at the results of a horse race. Sometimes a horse finishes first by 20 lengths, sometimes only by a nose. In either case, the horse still wins, but, statistically speaking, the distance between points matters.
3) Interval-- An interval level of measurement provides a rank order AND equal distance between the points, but has no real "zero." The Fahrenheit scale of temperature is an every day example. The difference between 70 and 80 is the same as the difference between 20 and 30 (10 degrees). The zero on the scale, however, is not a TRUE zero, since the measurement does not indicate a lack of the concept, but, rather, just one point on the continuum.
4) Ratio-- A ratio level of measurement provides rank order, equal distance AND has a true zero-- which indicates none of the concept. For example, if I'm measuring "TV viewing" in hours for the past day and I observe that you did not watch even one moment of TV during that time, then I can record "0."
Here are some sample variables. Indicate, based on how the variables is being measured, the level of measurement:
1. What is your gender?
1. Female
2. Male
3. Other: __________
2. What color hair does the cartoon character have?
1. Blonde
2. Brown
3. Black
4. Red
5. Other: ____________
3. Indicate your level of agreement with the following statement:
College is too expensive.
5-Strongly Agree 4- Agree 3- Neutral 2-Disagree 1-Strongly Disagree
4. Please indicate total your income for the year 2009: _________________
5. Please indicate your favorite electronic media from favorite (5) to least favorite (1):
a. Internet
b. Television
c. Music
d. Video games
e. Movies
Post your answers here.
Jack
Two big concepts-- Reliability and Validity
Without question, in order to understand effective social science research (or any kind of scientific research), you have to understand the concepts of "validity" and "reliability."
To put it simply, "validity" refers to whether a measure actually measures what you purport to be measuring.
For example, if you create a concept called "television use" and then decide to measure it by asking people how many TVs they own, that MIGHT be an indicator of how much TV they watch, but you definitely have some validity problems, right? Why? (post your thoughts as a comment to this entry).
It would probably be better to measure the concept by asking people how many hours of TV per day they watch, on average (or better still, you might have them go hour by hour thinking only of yesterday and to report what they watched during the day).
I'm sure you can see that using a measurement like this is better than just asking how many TVs someone owns...
Reliability, meanwhile, simply refers to how often you can repeat the measurement and get the same result.
Let's take an example and put the two concepts together--
Suppose you have a digital bathroom scale and you step up on it and weigh yourself and it reads "145 lbs."
Now let's suppose you repeat that process ten times in a row and you get results like this--
1. 145
2. 144
3. 144.5
4. 145
5. 145.1
6. 144.5
7. 144.8
8. 145.2
9. 145
10. 144
Acting reasonably, we should see these results and say "this scale has come pretty close to giving me the same reading 10 times in a row, so I conclude it's reliable." If so, its "reliability" is strong and is not in question.
What we do not know, however, is if the scale is RIGHT. What if it's wrong by, say, 10 lbs. and you REALLy weigh closer to 155 lbs.?
The scale's readings are reliable, but we can't say for sure if the scale is valid.
To confirm that the scale really is measuring pounds, we might "test" it by weighing other items whose weights we already know. For example, a 10 lb. bag of potatoes, a weight (from a weight room) of 25 lbs., a 50 lb. bag of rock salt, the official rod of steel used as the standard to determine a "pound" and so on.
Now, if we weigh all those items and each time the scale gives us readings that are really close to what we should expect, we can conclude that the scale is indeed valid.
Get it?
Let's practice. Below are several sample ways of measuring certain media concepts. Indicate to me, as comments, whether you think the measurements are valid (valid or NOT valid) and why or why not. In each case, one measurement is more valid than the other. Identify the valid measurement and indicate why the other is NOT valid.
Concept Measurement
media use
A) how much time you spend reading a daily newspaper
B) combination of how much time you spend on all media
exposure to TV violence
A) how many hours of TV you watch each day
B) how many hours of TV you watch each day broken up by TYPE of tv show watched (coded for content)
internet use
A) how many times a day you check Facebook
B) how many hours per day you spend online via computer or other handheld device
You may be thinking to yourself-- "this is pretty obvious stuff," but you'd be surprised at the number of reports professionals put together that suffer from problems of validity.
To put it simply, "validity" refers to whether a measure actually measures what you purport to be measuring.
For example, if you create a concept called "television use" and then decide to measure it by asking people how many TVs they own, that MIGHT be an indicator of how much TV they watch, but you definitely have some validity problems, right? Why? (post your thoughts as a comment to this entry).
It would probably be better to measure the concept by asking people how many hours of TV per day they watch, on average (or better still, you might have them go hour by hour thinking only of yesterday and to report what they watched during the day).
I'm sure you can see that using a measurement like this is better than just asking how many TVs someone owns...
Reliability, meanwhile, simply refers to how often you can repeat the measurement and get the same result.
Let's take an example and put the two concepts together--
Suppose you have a digital bathroom scale and you step up on it and weigh yourself and it reads "145 lbs."
Now let's suppose you repeat that process ten times in a row and you get results like this--
1. 145
2. 144
3. 144.5
4. 145
5. 145.1
6. 144.5
7. 144.8
8. 145.2
9. 145
10. 144
Acting reasonably, we should see these results and say "this scale has come pretty close to giving me the same reading 10 times in a row, so I conclude it's reliable." If so, its "reliability" is strong and is not in question.
What we do not know, however, is if the scale is RIGHT. What if it's wrong by, say, 10 lbs. and you REALLy weigh closer to 155 lbs.?
The scale's readings are reliable, but we can't say for sure if the scale is valid.
To confirm that the scale really is measuring pounds, we might "test" it by weighing other items whose weights we already know. For example, a 10 lb. bag of potatoes, a weight (from a weight room) of 25 lbs., a 50 lb. bag of rock salt, the official rod of steel used as the standard to determine a "pound" and so on.
Now, if we weigh all those items and each time the scale gives us readings that are really close to what we should expect, we can conclude that the scale is indeed valid.
Get it?
Let's practice. Below are several sample ways of measuring certain media concepts. Indicate to me, as comments, whether you think the measurements are valid (valid or NOT valid) and why or why not. In each case, one measurement is more valid than the other. Identify the valid measurement and indicate why the other is NOT valid.
Concept Measurement
media use
A) how much time you spend reading a daily newspaper
B) combination of how much time you spend on all media
exposure to TV violence
A) how many hours of TV you watch each day
B) how many hours of TV you watch each day broken up by TYPE of tv show watched (coded for content)
internet use
A) how many times a day you check Facebook
B) how many hours per day you spend online via computer or other handheld device
You may be thinking to yourself-- "this is pretty obvious stuff," but you'd be surprised at the number of reports professionals put together that suffer from problems of validity.
Understanding IV and DV
Ok, so in a nutshell, here's how quantitative research works (we'll talk about qualitative later)--
First, you come up with an interesting question that you'd like to answer. In other words, you come up with concepts to see if they're related. Maybe you're interested in the relationship between TV viewing and obesity in kids; the portrayal of models of fashion magazines and real-life body image among females (or males); video games and reflexes; video games and violence; sexual content on TV and real-life sexual behavior; rap/rock lyrics and attitudes toward women; and so on. The possibilities are, literally, endless.
Then, using your own observations, thoughts, opinions, etc., you decide which way you believe the relationship goes. You come up with a declarative statement like "kids who watch a lot of TV get fat."
You believe this to be true for whatever reason. This is known as your "theoretical rationale." Why do you think it might be true? As long as it makes sense (face validity), you're probably on to something.
For example, you might say-- "well, kids who watch a lot of TV are spending time watching TV INSTEAD of running around and playing outside, so they're probably not getting a lot of exercise, so they're not burning as many calories. Also, it seems likely that kids watching TV are more likely to mindlessly snack than are kids playing kickball or some other activity. So, it seems to me that kids watching TV burn less calories and consume more calories, so it makes sense that this might lead to more childhood obesity."
Makes sense to me.
Every premise has a theoretical rationale.
We need to refine it, however, and form a hypothesis. A hypothesis is simply a declarative, testable statement that examines the relationship between variables.
Variables are either independent (IV) or dependent (DV).
An IV is one that isn't changed... A DV, however, is the one that changes. We measure DVs.
For example, if I developed a hypothesis on the tv-obesity topic, I might come up with something like this:
H1: The more TV a child watches, the more likely the child is overweight.
In this one simple hypothesis, we have three concepts that we need to identify. What do we mean by "child," "TV watching," and "overweight?"
The conceptual definition is the dictionary definition of the concept, and how we plan on "measuring" that concept is called the "operational" definition.
For example, TV watching is defined as the number of hours, on average, someone watches TV per day (conceptual definition). To measure this, we had children circle the TV shows they watched "yesterday" from a grid provided to them (operational definition).
Get it? You'd have to do this for each concept.
In terms of variables, in our first hypothesis, the IV is "TV watching" and the DV is "weight." We are suggesting, at least in this hypothesis, that an individual's "weight" will change based on how much TV he/she watches. Since we're suggesting that TV viewing can "influence" weight, weight is the DV.
To make it easier for us, most scholars follow the form of putting your IV first and your DV second in any hypothesis.
Then, once we've got this all figured out, we need to figure out how the heck we would "test" this hypothesis. The "test" is the statistical method used to figure out if the relationship between the variables is significant.
In this case, the IV is "ratio" and the DV is also "ratio." When we have two ratio variables, we always use "correlation" as the statistical test.
When the IV is "nominal" and the DV is "nominal," we use chi-square.
When the IV is "nominal" and the DV is "interval/ratio," we use t-test.
When the IV is "interval/ratio" and the DV is "interval/ratio," we use correlation.
(In this class, we won't discuss what to use if the IV is "interval/ratio," and the DV is "nominal" -- logistical regression).
We'll talk more about the tests later in the course, but it's a good idea to know which test is used in which circumstance...
Obviously, you don't need to know what these statistical tests are yet, but we'll get to them next week.
Per usual, let me know if you have any questions, you crazy quant-heads.
Peace,
Jack
First, you come up with an interesting question that you'd like to answer. In other words, you come up with concepts to see if they're related. Maybe you're interested in the relationship between TV viewing and obesity in kids; the portrayal of models of fashion magazines and real-life body image among females (or males); video games and reflexes; video games and violence; sexual content on TV and real-life sexual behavior; rap/rock lyrics and attitudes toward women; and so on. The possibilities are, literally, endless.
Then, using your own observations, thoughts, opinions, etc., you decide which way you believe the relationship goes. You come up with a declarative statement like "kids who watch a lot of TV get fat."
You believe this to be true for whatever reason. This is known as your "theoretical rationale." Why do you think it might be true? As long as it makes sense (face validity), you're probably on to something.
For example, you might say-- "well, kids who watch a lot of TV are spending time watching TV INSTEAD of running around and playing outside, so they're probably not getting a lot of exercise, so they're not burning as many calories. Also, it seems likely that kids watching TV are more likely to mindlessly snack than are kids playing kickball or some other activity. So, it seems to me that kids watching TV burn less calories and consume more calories, so it makes sense that this might lead to more childhood obesity."
Makes sense to me.
Every premise has a theoretical rationale.
We need to refine it, however, and form a hypothesis. A hypothesis is simply a declarative, testable statement that examines the relationship between variables.
Variables are either independent (IV) or dependent (DV).
An IV is one that isn't changed... A DV, however, is the one that changes. We measure DVs.
For example, if I developed a hypothesis on the tv-obesity topic, I might come up with something like this:
H1: The more TV a child watches, the more likely the child is overweight.
In this one simple hypothesis, we have three concepts that we need to identify. What do we mean by "child," "TV watching," and "overweight?"
The conceptual definition is the dictionary definition of the concept, and how we plan on "measuring" that concept is called the "operational" definition.
For example, TV watching is defined as the number of hours, on average, someone watches TV per day (conceptual definition). To measure this, we had children circle the TV shows they watched "yesterday" from a grid provided to them (operational definition).
Get it? You'd have to do this for each concept.
In terms of variables, in our first hypothesis, the IV is "TV watching" and the DV is "weight." We are suggesting, at least in this hypothesis, that an individual's "weight" will change based on how much TV he/she watches. Since we're suggesting that TV viewing can "influence" weight, weight is the DV.
To make it easier for us, most scholars follow the form of putting your IV first and your DV second in any hypothesis.
Then, once we've got this all figured out, we need to figure out how the heck we would "test" this hypothesis. The "test" is the statistical method used to figure out if the relationship between the variables is significant.
In this case, the IV is "ratio" and the DV is also "ratio." When we have two ratio variables, we always use "correlation" as the statistical test.
When the IV is "nominal" and the DV is "nominal," we use chi-square.
When the IV is "nominal" and the DV is "interval/ratio," we use t-test.
When the IV is "interval/ratio" and the DV is "interval/ratio," we use correlation.
(In this class, we won't discuss what to use if the IV is "interval/ratio," and the DV is "nominal" -- logistical regression).
We'll talk more about the tests later in the course, but it's a good idea to know which test is used in which circumstance...
Obviously, you don't need to know what these statistical tests are yet, but we'll get to them next week.
Per usual, let me know if you have any questions, you crazy quant-heads.
Peace,
Jack
Overview of MMRM
Greetings--
Below is a brief overview of quantitative research. In this class, we'll cover three major quantitative methodologies-- 1) surveys, 2) experiments, and 3) content analysis, and the three major qualitative methods-- 1) interviews, 2) focus groups, 3) participant-observation.
When it comes to social science, there are two approaches-- qualitative and quantitative. While both approaches use the scientific method, a qualitative researcher believes that humans and their behaviors are far too "messy" and complex to quantify and that it is silly to even try. Instead, qualitative researchers believe that any understanding of the human condition is best ascertained by talking to people and searching for identifiable patterns among their comments.
A quantitative researcher, by comparison, believes that if a concept exists, then it can be measured, and, conversely, if it can't be measured, then it doesn't exist.
For example, if we were to take the concept of "romantic love," an abstract concept to be sure, a quantitative researcher would say that if "love" is real and exists, then it can be measured. He/she might begin by identifying indicators of romantic love and they might include items like--
amount of time spent with the other person
level of affection
level of emotional connection
level of desire to spend time with the other person
phsyiological responses when with the other person
willingness to do things for the other person
level of communication with the other person (talking in person, snapchat, via cell, texting, email, IMing, facebook, skype, etc.)
sexual relationship
level of commitment
and so on...
The point is this- a quantitative researcher might be able to say "If I discover that a person has a high level of affection for someone, feels emotionally connected, wants to spend a lot of time with that person, experiences physiological symptoms when in the presence of the other person (heart rate increase, general feelings of happiness, goose bumps, etc.), wants to do things for that person, spends a lot of time communicating with him/her, has a healthy sexual relationship with him/her, and is committed for the long haul, and so on, then that person is 'in love.'"
Get it?
If there's one thing I want you to get out of this class, it's this-- quantitative research is practical and is a wonderful tool we have to answer some of life's most interesting questions.
So let's begin by taking a quick look at an overview of the quantitative approach:
Quantitative research overview
Quantitative research usually is designed to produce estimates of the prevalence of knowledge, attitudes, opinions, behaviors, and other characteristics of a defined population group, whether the U.S. population as a whole or some subgroup. Most quantitative research attempts to do this by using various approaches to randomly selecting a representative sample of the audience of interest, such as random household surveys, random digit-dialing telephone methodologies, or random selection of names from voter lists or other lists believed to be inclusive of all members of the audience (for example, membership lists, motor vehicle registrations, school enrollment lists).
These and other approaches can involve complex decisions about the comprehensiveness of the data from which to sample, whether the sampling should be "stratified" (a process in which attempts are made to represent different segments of the overall population of interest, such as by gender, age group, or household income), whether adequate numbers of people can be sampled from which to reliably make population estimates, as well as specific ways in which "random selection" is actually accomplished.
After a representative sample is selected, surveys are then administered to the sample via one or more modalities, including in-person, telephone, or mail, and, more recently, the Internet. Each of these methods presents technical challenges, among them ensuring that all members of the sample have equal access to the chosen modality so as to not introduce a sampling bias. Another major challenge is to achieve high response rates, so that the results are not potentially biased by unknown characteristics of either the "responders" or "nonresponders."
Because of the nature of this research, and its use of relatively high numbers of respondents, surveys often use a limited number of questions that are presented in a consistent way to all respondents. Approaches to questionnaire development and phrasing can vary in rigor from internal professional review and discussion to formal cognitive laboratory testing with potential respondents.
To facilitate data entry and analysis, most questions will be structured in a "closed-ended" format, which limits the respondents to making a choice between two or more predetermined alternative responses. "Open-ended" formats also can be used in this research. If responses given by respondents do not fall within a limited number of categories or themes that can be discerned by either software or analysis by the researcher, however, they can limit the value and generalizability of the findings.
Finally, the other defining feature of quantitative research is the statistical analysis of the responses and reporting of these data in summary form. Statistical analysis can range from simple frequencies, percentages, and cross-tabulations for selected subgroups (such as age group, income, race/ethnicity) to complex analyses that may try to explain how various characteristics of the population or subgroups relate to one another (for example, what characteristics explain differences between people who engage in a healthy behavior versus those who do not).
The results of these analyses are then used to draw conclusions about the prevalence of the measured characteristics in the larger population of interest. These results are often expressed using confidence intervals or p-values to gauge the level of certainty that the reported results may in fact reflect the population as a whole.
Benefits of quantitative research
The most important benefit of well-designed and well-implemented quantitative research is that it can give planners of communications programs fairly reliable information about the prevalence of certain characteristics among their audience. Quantitative research that is conducted on a periodic basis also can track the effects of the program on targeted knowledge, attitudes, and/or behavioral change objectives. Quantitative methods also can be used to determine if the results of qualitative research are valid for the larger population.
Limitations of quantitative research
Researchers must consider several limitations of quantitative research before making a decision to conduct audience research using this family of approaches. The more important limitations are:
* Such approaches usually are resource intensive and can take several weeks to many months to design, implement, and analyze, thus extending the time needed to incorporate audience-based research into program planning. One option to address this limitation is to add questions to ongoing omnibus marketing and opinion-sampling surveys conducted by commercial entities. These results can be turned around much more quickly. How these firms construct their samples, what modalities they use for interviewing, and how they achieve high response rates all need to be explored, however, to assure the quality of the information provided to the planners.
* Quantitative research also requires skills in sampling design issues, sampling methodologies, survey design, statistical techniques, and how they are all applied in a communications research context. The extent to which these skills are used in planning and carrying out a quantitative study determines both the quality of the data and their generalizability to the total population.
* The structure of most surveys limits the number of questions that can be asked, the variety of responses that respondents can provide, the time each respondent has to answer questions (15–20 minutes is what many surveys aim for to minimize respondent burden and to maximize full completion of the survey), and any type of interactive process with or among respondents. Thus the data are limited in the amount and richness of audience information that can be fed into the program and message design processes.
Below is a brief overview of quantitative research. In this class, we'll cover three major quantitative methodologies-- 1) surveys, 2) experiments, and 3) content analysis, and the three major qualitative methods-- 1) interviews, 2) focus groups, 3) participant-observation.
When it comes to social science, there are two approaches-- qualitative and quantitative. While both approaches use the scientific method, a qualitative researcher believes that humans and their behaviors are far too "messy" and complex to quantify and that it is silly to even try. Instead, qualitative researchers believe that any understanding of the human condition is best ascertained by talking to people and searching for identifiable patterns among their comments.
A quantitative researcher, by comparison, believes that if a concept exists, then it can be measured, and, conversely, if it can't be measured, then it doesn't exist.
For example, if we were to take the concept of "romantic love," an abstract concept to be sure, a quantitative researcher would say that if "love" is real and exists, then it can be measured. He/she might begin by identifying indicators of romantic love and they might include items like--
amount of time spent with the other person
level of affection
level of emotional connection
level of desire to spend time with the other person
phsyiological responses when with the other person
willingness to do things for the other person
level of communication with the other person (talking in person, snapchat, via cell, texting, email, IMing, facebook, skype, etc.)
sexual relationship
level of commitment
and so on...
The point is this- a quantitative researcher might be able to say "If I discover that a person has a high level of affection for someone, feels emotionally connected, wants to spend a lot of time with that person, experiences physiological symptoms when in the presence of the other person (heart rate increase, general feelings of happiness, goose bumps, etc.), wants to do things for that person, spends a lot of time communicating with him/her, has a healthy sexual relationship with him/her, and is committed for the long haul, and so on, then that person is 'in love.'"
Get it?
If there's one thing I want you to get out of this class, it's this-- quantitative research is practical and is a wonderful tool we have to answer some of life's most interesting questions.
So let's begin by taking a quick look at an overview of the quantitative approach:
Quantitative research overview
Quantitative research usually is designed to produce estimates of the prevalence of knowledge, attitudes, opinions, behaviors, and other characteristics of a defined population group, whether the U.S. population as a whole or some subgroup. Most quantitative research attempts to do this by using various approaches to randomly selecting a representative sample of the audience of interest, such as random household surveys, random digit-dialing telephone methodologies, or random selection of names from voter lists or other lists believed to be inclusive of all members of the audience (for example, membership lists, motor vehicle registrations, school enrollment lists).
These and other approaches can involve complex decisions about the comprehensiveness of the data from which to sample, whether the sampling should be "stratified" (a process in which attempts are made to represent different segments of the overall population of interest, such as by gender, age group, or household income), whether adequate numbers of people can be sampled from which to reliably make population estimates, as well as specific ways in which "random selection" is actually accomplished.
After a representative sample is selected, surveys are then administered to the sample via one or more modalities, including in-person, telephone, or mail, and, more recently, the Internet. Each of these methods presents technical challenges, among them ensuring that all members of the sample have equal access to the chosen modality so as to not introduce a sampling bias. Another major challenge is to achieve high response rates, so that the results are not potentially biased by unknown characteristics of either the "responders" or "nonresponders."
Because of the nature of this research, and its use of relatively high numbers of respondents, surveys often use a limited number of questions that are presented in a consistent way to all respondents. Approaches to questionnaire development and phrasing can vary in rigor from internal professional review and discussion to formal cognitive laboratory testing with potential respondents.
To facilitate data entry and analysis, most questions will be structured in a "closed-ended" format, which limits the respondents to making a choice between two or more predetermined alternative responses. "Open-ended" formats also can be used in this research. If responses given by respondents do not fall within a limited number of categories or themes that can be discerned by either software or analysis by the researcher, however, they can limit the value and generalizability of the findings.
Finally, the other defining feature of quantitative research is the statistical analysis of the responses and reporting of these data in summary form. Statistical analysis can range from simple frequencies, percentages, and cross-tabulations for selected subgroups (such as age group, income, race/ethnicity) to complex analyses that may try to explain how various characteristics of the population or subgroups relate to one another (for example, what characteristics explain differences between people who engage in a healthy behavior versus those who do not).
The results of these analyses are then used to draw conclusions about the prevalence of the measured characteristics in the larger population of interest. These results are often expressed using confidence intervals or p-values to gauge the level of certainty that the reported results may in fact reflect the population as a whole.
Benefits of quantitative research
The most important benefit of well-designed and well-implemented quantitative research is that it can give planners of communications programs fairly reliable information about the prevalence of certain characteristics among their audience. Quantitative research that is conducted on a periodic basis also can track the effects of the program on targeted knowledge, attitudes, and/or behavioral change objectives. Quantitative methods also can be used to determine if the results of qualitative research are valid for the larger population.
Limitations of quantitative research
Researchers must consider several limitations of quantitative research before making a decision to conduct audience research using this family of approaches. The more important limitations are:
* Such approaches usually are resource intensive and can take several weeks to many months to design, implement, and analyze, thus extending the time needed to incorporate audience-based research into program planning. One option to address this limitation is to add questions to ongoing omnibus marketing and opinion-sampling surveys conducted by commercial entities. These results can be turned around much more quickly. How these firms construct their samples, what modalities they use for interviewing, and how they achieve high response rates all need to be explored, however, to assure the quality of the information provided to the planners.
* Quantitative research also requires skills in sampling design issues, sampling methodologies, survey design, statistical techniques, and how they are all applied in a communications research context. The extent to which these skills are used in planning and carrying out a quantitative study determines both the quality of the data and their generalizability to the total population.
* The structure of most surveys limits the number of questions that can be asked, the variety of responses that respondents can provide, the time each respondent has to answer questions (15–20 minutes is what many surveys aim for to minimize respondent burden and to maximize full completion of the survey), and any type of interactive process with or among respondents. Thus the data are limited in the amount and richness of audience information that can be fed into the program and message design processes.
Subscribe to:
Posts (Atom)