Report by firstname.lastname@example.org (2013-05-02)
The survey was conducted from 2013-04-24 until 2013-05-02 using LimeSurvey. The survey consisted of two pages. The first was a short set of demographic questions: country, sex, age, and whether the respondant was current user of OpenOffice. If the user stated he was a current OpenOffice user follow-up questions asked how long he has been using OpenOffice, on what primary platform, and whether at work, home or school (multiple choices allowed).
The second page of the survey presented each of the 40 logo proposals and asked the respondant to rate each logo on a 5-point Likert scale: Strongly Dislike, Dislike, Neither like nor dislike, Like, Strongly Like. Logos were normalized to 400 pixel width and were presented in random order for each respondant, to avoid any ordering bias. A comment field was also available for each logo.
As a control, the current logo from Apache OpenOffice 3.4.1 was randomly inserted into the survey as well.
5028 responses were received.
Responses were received from 126 countries.
|Country||Number of responses|
|Bosnia and Herzegovina||5|
|British Indian Ocean Territory||1|
|Iran, Islamic Republic of||11|
|Korea, Democratic People's Republic||3|
|Korea, Republic of||6|
|Lao People's Democratic Republic||0|
|Libyan Arab Jamahiriya||1|
|Saint Kitts and Nevis||1|
|Syrian Arab Republic||4|
|Turks and Caicos Islands||5|
|Trinidad and Tobago||1|
|United Arab Emirates||3|
|Virgin Islands, U.S.||1|
|Sex||Number of Responses|
As the following histogram shows, many respondants claimed to be 70, 80 or 90 year old. As described further in the Data Issues section, we suspect that many users entered their two-digit birth year rather than their age.
3680 respondants said Yes, 564 said No.
|Place||Number of Responses|
|Operating System||Number of Responses|
|Logo||Image||Strong Dislike||Dislike||Neither Like nor Dislike||Like||Strongly Like|
To make some sense of the raw counts in the table above we will need to do some exploratory analysis. The following analysis and charts (and the charts above) were done using the open source R statistical package.
The logo preference questions were in the form of a 5-point Likert scale, from Strongly Dislike to Strongly Like. Whenever dealing with a Likert scale the question naturally arises: can we treat the data as interval data? Or is it only ordinal? Ordinal data expresses a rank of sentiment only. It monotonically increases with the scale. Interval data additionally implies that the scale is linear, that for example, the distance from Dislike to Neutral is the same as the distance from Neutral to Like, or from Like to Strong Like.
With an ordinal interpretation we can look at histograms (counts of scores), at the mode (most frequent response), median (the middle value) and the variation ratio (fraction of scores not in the mode).
With an interval interpretation we would assign each point on the scale a numeric value, e.g., 1 for Strongly Dislike to 5 for Strongly Like. Then we could take these scores and calculate means and standard deviations.
In our analysis we'll mix ordinal and interval intepretations.
The following issues should be noted:
The following dot chart shows the most freqent score (the Mode) for each logo, where 1=Strongly Dislike, 2=Dislike, 3=Neither Like nor Dislike, 4=Like and 5=Strongly Like. Although no logo received mainly Strong Likes, logos 5, 12, 13, 24, 26, 27, 28, 29, 30, 31, 34 and 36 all received mainly Like's (score = 4).
Next we apply an interval interpretation to the data, and calculate the numeric mean of each logo's score, 1=Strongly Dislike, 2=Dislike, 3=Neither Like nor Dislike, 4=Like and 5=Strongly Like. In this plot, logos 34 and 36 are at the top.
Another way to approach the data is to look for the logos with the most enthusiastic support, i.e., the greatest number of Strongly Like scores. When we chart this we see that logos 04 and 15 lead the pack, followed by 26, 36 and 34.
Knowing the mode or mean tells only part of the story. In particular, it does not tell us how spread out the responses are. So an average score of 3.0 might come about because the logo was given all responses of Neither Like nor Dislike. Or it could be because it received an equal number of Strong Likes and Strong Dislikes. If we plot the means and the standard deviations we can pick out which logos had high scores, but also high consensus (low standard deviation) around those scores.
In the following chart the standard deviation increases from left to right and the mean increases from top to bottom. The best location then is in the upper left corner, where we see a grouping of logos 34, 36, 31, 05, 12, 13 and 28. The horizontal line at Mean = 3.0 marks the "Neither Like nor Dislike" line. Logos above that line were liked more than disliked, while those below were net disliked.
Since we earlier noted that women were underrepresented in this survey, it is important to check to see if this introduced any bias in the results. One way to answer that question is to plot the mean score per logo for men against the scores for women. If the scores for men were identical, they would all lie along the diagonal line plotted in the following chart. We're looking for any significant deviations from that line. Generally the results are not too far apart, but it should be noted that the top ranked logo for women was 36, while men rated 34 the highest.
Similarly we want to look at how preference scores differed for those who said they were OpenOffice users versus those who are not. As the following chart shows, while there are a few logos that rated noticably higher for Non-users than users, at the top of the scale there were no significant differences.
We randomly inserted the current OpenOffice logo into the survey as a control. It scored very highly, comparable to logo 34 and 36.
A few of the logos, predominately the so-called flat logos, did very well in this survey, in particular 34, 36, 05, 28, 12, 13 and 31. We should look into the comments for these logos (which are linked to here, along with the score given by the user who submitted the comment) and see if there is some way we can encorporate feedback to improve. And logos 15 and 04, although on average did not hit the top ranks, did receive so many enthusiastic Strongly Like scores, that it would be worth looking at the comments there as well, to see if there is some easy way to raise their scores.