Advertisements

Brick by brick: News about FontStruct


Adjusting the rating system

News | | September 11th, 2009

We get a lot of positive feedback from FontStructors through email, on FontStruct threads and specially on Twitter. It feels great to give people something that makes them feel happy and creative. Thousands of FontStructors who voluntarily choose to share their work with the world are discovering this feeling too.

Of course, among all praises there are also feature requests, occasional bug reports and complaints. Interestingly, most complaints we get are targeted at one particular feature in FontStruct: the rating system.

The rating feature allows users to give grades – from 1 to 10 – to FontStructions created by other users. The idea is to have a parallel, user-driven quality control system complementary to Top Picks and Featured FontStructions (which are distributed by the FontStruct staff). We find this a useful and important feature to have; it invites participation and increases the self-organization powers of the community.

Why FontStructors complain about rating

FontStructors complaints are directed mainly to the great influence of “FontTrolls” – nasty creatures who appear to have great fun in giving very low grades to others – in the ratings. Several suggestions for improvements have been made by FontStructors (see the the FontTroll and FontMoot threads for examples), from the total removal of the rating feature to the acceptance of the rating system as it is.

Some FontStructors have also raised their voice against the recent inflation of 10s, which is as damaging to the system as the low ratings distributed by trolls – both distort the scale and take away the fine grading of intermediary values.

Looking at the rating data

We agreed that the rating system could be improved, and decided to work on it. Our first action was to look at the rating data to get a better idea of what was happening. The result of this research was very interesting, and gave us a solid foundation to plan and implement the necessary adjustments.

The following is a brief summary of our findings:

FontStructions – What gets voted?

data_fontstructions_00

The chart shows that Featured FontStructions get more and higher votes than Top Picks, which get more and higher votes than regular public FontStructions.

The more exposure a FontStruction gets, the more votes and the highest the ratings it receives.

FontStructors – Who votes?

data_fontstructors_00

Among all registered FontStructors, only a small minority (1,6%) makes use of the rating feature. Among the voters, FontStructors with Top Picks vote more than regular FontStructors, who vote more than users who registered but never created a FontStruction.

FontStructors with Top Picks are more demanding and vote in average lower than regular FontStructors, who vote in average lower than users who never created a FontStruction.

How do trolls behave?

Looking at the voting data we were able to identify three main “species” of trolls:

1. Down-voting trolls – The classic sort. We identified 9 “mini-trolls” (who have cast 10 or more votes with an average rating of less than 2), and 4 “mega-trolls” (who have cast 1000 or more votes with an average rating of less than 2).

2. Positrolls – Users who always and only award ratings of 10. We’ve found several “Robin Hood” accounts which appear to have been created solely for the purpose of spreading love and counterbalancing trollery.

3. Naughty Positrolls – Accounts used to boost the ratings of individual FontStructors. There were several of them!

* * *

Based on the data we gathered, we began to draw conclusions and think about what could be done to improve the system.

In Rob Meek’s words:

Some people get very upset by the ratings system, in particular by low votes (from “trolls”) which can seem unfair. Some people are more sensitive than others, and there are also plenty of people who are happy with the ratings system as it is. Many people have also invested a lot of time and thought in rating hundreds of FontStructions so I don’t think we should throw the current system out the window.

Looking at how people vote, it is clear that people have completely different standards and frames of reference for voting and understand the voting system differently: Some try and rate a FontStruction in terms of its value in the context of FontStruct while some rate it for its value in a broad typographic context. Some people obviously dislike the trend for high-resolution designs and vote accordingly. Others see voting solely as a way to encourage and spread positive karma. One person’s 7 is another’s 5. Many people may vote correctively i.e. they don’t just give a design the rating they feel is right, they try and “correct” the existing rating with a more extreme vote – I think this may explain some trolling. (…)

Looking at the data I think the best way to improve the system, and get meaningful ratings, is to pursue these two main goals:

  1. encourage and help people to vote more often. it’s simply low vote counts that make trolling significant.
  2. relativize the dominance of the linear gallery listings which make ratings so important and represent a catch-22: You need a high rating to get to the top, and you need to be at the top to get a high rating.

It was decided to invalidate votes from “Naughty Positrolls” (accounts used to boost the ratings of individual FontStructors). Votes from mega-trolls were left untouched, since they vote differently from each other, and most importantly they do sometimes vote high (they just seem to have a very strict criteria for quality). We also decided not to filter out statistically aberrant votes (i.e. the one low voter when everyone else votes high), as this would be plain censorship.

* * *

Ladies and gentleman — Without further delay, here are the long awaited changes to the rating system:

Vote indicator

The first visible change for voters is a grey check mark next to the ratings bar (the stars visible on FontStruction pages and in the gallery), meaning that the FontStruction has already received a vote from you. This helps to avoid repeating votes and stimulates users to rate different FontStructions.

voting_01

rating_01

New balancing formula

Balanced rating is probably the most important new feature being introduced today. The average rating values of a FontStruction are now calculated with a new balancing formula, which “pulls” ratings towards the system wide average. FontStructions with only a few votes can have neither very high nor very low ratings. As they get more votes, the balanced rating gets closer and closer to the true average. In general, for high-rated FontStructions, the “balanced rating” will be lower than the true average. For low-rated fonts with only a few votes, the “balanced” rating will be higher than the true average.

rating_00

The old average rating is still there for comparison — it is displayed via mouse-over on the ratings, and appears as an alternative sorting parameter in the gallery.

Vote breakdown

The new vote breakdown is an alternative representation of the votes received by a FontStruction, showing the distribution of grades in graph. It can be accessed by clicking on the rating of a FontStruction.

rating_01

The sample image shows the typical distribution of votes in a Top Pick – an overwhelming amount of 10s, but also several low votes from trolls or critics.

Hidden gems

Last but not least, the gallery now has a new category: Hidden Gems.

HiddenGems_00

Hidden Gems are special FontStructions which are asking for your attention and your votes. Accessible from the main Gallery menu, Hidden Gems are a random selection of FontStructions which (A) you have not voted on and (B) seem to have been neglected in some way – maybe they’ve been downloaded frequently but have few received votes, or they are a Top Pick with a low rating.

Like FreeTags, Hidden Gems bring an element of surprise to the browsing of FontStructions.

* * *

Conclusions

We are excited to launch the new features and to pass the result of all this work to you.

We hope that these small adjustments will make it more fun to vote and increase the percentage of voting users. With more users voting and the new balanced formula, the quality of the rating data will only get better.

Finally, we have plans to add further ways of celebrating good designs, and involving the community in that process. Have fun & stay tuned!


36 Comments

  1. jmarquez

    Great Job!

    Comment by jmarquez — September 11, 2009 #


  2. Em

    Wow, not checked yet, but it looks like you did a great job!

    Thanks for working on this feature!

    Comment by Em — September 11, 2009 #


  3. Magic Sam

    I don´t know if I like the new rating system…I think the old system isn´t perfect, but ok.

    The new balancing formula votes Fontstructions with high ratings down, and FS with low ratings higher. But why? For instance, if 4 people rate 10 and one votes 8, the balanced rating will be around 8 ??

    Besides, the new voting system seems to malfunction: most of my (newer) fonts have an average rating of 8.* and a balanced rating of 0.0. I don´t get it…

    But I like the Hidden Gems!

    Comment by Magic Sam — September 11, 2009 #


  4. p2pnut

    An impressively thorough review of the rating system, resulting in what appears to be a very good solution.

    I confess that I have on several occasions given a 10 to work that didn’t fully deserve it – just to redress the effect of “down-voting trolls”.

    A negative vote doesn’t worry me, I’m a mature person who indulges in this wonderful program simply for his own pleasure. However it concerns me that these mindless trolls could knock the confidence of younger, up-and-coming ‘structors. It must also be galling for those in the FS community who are professionals in the design world to have their hard work denigrated in this way.

    You appear to have the ability to identify these down-voters; is there any way in which they can be sanctioned? Possibly by first receiving a warning then, if their negative behavious persists, by suspension of their voting priviledge.

    Comment by p2pnut — September 11, 2009 #


  5. aphoria

    I’m willing to wait and see, but the new rating system seems overly complex.

    To try it out, I found a font with 0 votes and gave it an 8. The balanced rating then showed 6. That doesn’t seem right to me.

    The Hidden Gems feature is cool though!

    Comment by aphoria — September 11, 2009 #


  6. Magic Sam

    @aphoria: That´s what I mean.
    Just take a look at “Triple DECOr” by P2pnut, for instance. There are two votes: a 10 and a 9. The balanced rating shows a 7…strange.

    Comment by Magic Sam — September 11, 2009 #


  7. p2pnut

    LOL I just had a quick look at My FontStruct and now I appear to have quite a few showing 0 … but which have an average rating of around 7 or 8

    Maybe my earlier comment – that it appears to be a very good solution. – was a tad premature.

    Still, it’s early days and I’m sure the Staff Team will sort the system out in the fullness of time :)

    Comment by p2pnut — September 11, 2009 #


  8. funk_king

    thanks fsstaff for the time and effort put into improving the ratings system. the enhancements add access to additional info which is interesting. i guess we’ll have to wait to see how it pans out. i have tried to put this issue in a place that doesn’t impact my use of FS. i think anytime you allow someone else to criticize or vote on anything, you introduce many things – good and bad. personal bias, resentment, jealously, envy, to name a few; but also appreciation, admiration, acknowledgment.

    perhaps we have given to much attention to the font troll. when i created font troll it was a reaction to my observation – right or wrong – that my fonts were being downvoted intentionally for malicious reason. perhaps a bit paranoid, but still. in your overview of the new enhancements, i was quite entertained by the identification of new species of font troll – some which hit close to home :) have i upvoted based more upon who the FSor was vs. the overall font? yes. have i participated in defensive voting when i was pissed that a font had been overly downvoted? yes. these thing were done to defend my art.

    i really don’t care what anyone thinks about my fonts other than that they could be nice to dl and use. i really wish we could turn off the rating display if we don’t want to see it. to be affronted with this every time you view myfontsruct page is unnecessary and to me doesn’t necessarily promote something positive. show it on other views or if you select the option, otherwise, don’t show. i took a look at 1001fonts.com. they show a percentage and popularity – which to me doesn’t seem quite as mean as what we have here.

    if anything, i would like to see more stats on dls. perhaps giving the ability to see daily, monthly, and yearly views of dls for fonts. maybe a page showing say your top dls and then showing the count breakdowns by selected period. now that to me would be cool.

    but i very much appreciate what you guys have done and continue to do. i guess as much as we complain about the perceived pain of hurt feelings regarding being downvoted; i’m sure that pales in comparison to how you guys feel about us sometimes and our gripes. i for one am truly grateful for this great tool and the ability to express myself in ways i couldn’t do otherwise :) to me that’s a big deal and everything else is besides the point. peace :)

    Comment by funk_king — September 11, 2009 #


  9. afrojet

    Thanks team Fontstruct. Will definitely explore everything. On first blush, it’s a real eye opener to be able to drill down and look at those ratings graphs. Although I almost wish I had never seen ‘em – all those 1 votes – what a drag.

    Comment by afrojet — September 11, 2009 #


  10. Rob Meek (meek)

    Thx for the feedback everyone. I’m sure there will be more to come so I won’t give a lengthy response now. I think some changes will be more popular than others – the balanced rating likely to be the most contraversial. I’d like to stress that votes are recorded as they always have been, so the voting system itself hasn’t really changed – we’re just offering a new representation of the average vote. It’s a standard solution, used by many websites which have ratings system, called Bayesian ratings. It does take some getting used to, as it doesn’t make intuitive sense until a FontStruction gets a certain number of votes. On the first few votes, it’s a bit wierd.

    @p2pnut I’ll look at those zero rated FontStructions – that shouldn’t happen obviously. We’re not going to sanction any real people, and any analysis of voting patterns is done anonymously.

    @funk_king Interesting feedback. I’m sure a lot of people have similar experiences. Sometimes I would like to turn the ratings system off myself ;-)

    As Gustavo says in the article, one of our aims is to encourage people to vote more often – and to make that easier – this was the main motivation behind the ‘remember me’ checkbox on the sign in page. We believe that more votes will give more meaningful ratings.

    Comment by Rob Meek (meek) — September 11, 2009 #


  11. Stephen Coles (stewf)

    It feels great to see one of Rob’s early ideas (I think he and I were talking about this over a year ago) finally make it to FontStruct, and executed in such fine fashion. Like Rob says, this is not a ratings system replacement but simply the addition of more visibility into ratings.

    aphoria:

    To try it out, I found a font with 0 votes and gave it an 8. The balanced rating then showed 6.

    This is the intended result. The ratings are weighted to the middle (5) so that a single vote doesn’t have a drastic affect on the overall rating. Once a few more people give that FontStruction a high rating then it will move toward an 8 overall.

    Comment by Stephen Coles (stewf) — September 11, 2009 #


  12. cayo

    I’m sorry but I think it’s kind of fun how FS has coined so many terms lately: Font-trolls, Font-fairies, Robin Hood Accounts (LOL) etc. I mean we are developing our own FS slang here, in I lighter side of things.

    Still, I apreciate the effort taken by the FS crew to satisfy their FSors (err… us!) with a decent voting system.

    I agree with Funk_King with the hidden voting option (well, I agree with his whole exposure), because for it’s more influential to people actually outside of the FS bubble. Like when you show your FSions to friends who know nothing about trolling, but I can live with it. So I’m not going to ask for changes or make complaints, but rather share my point of view:

    I may be guilty of all charges exposed by Gustavo, but I have grown to understand apreciation (or dislike) of my fellow FSors more by their comments, than by the voting system.
    I mean, it’s nice when someone puts a 10, but it’s just a click. But when someone writes a comment (it could be just a simple “cool” or a nasty “i hate it”) they take some minutes to do so. Since time is money, it’s like getting paid.

    Anyways, I like to expose my gratitude for the effort taken by the FS crew to lighten up our feelings with such improvements. Thanks.

    Comment by cayo — September 11, 2009 #


  13. paul d hunt

    some things seem not quite right. i will use an example that involves one of my fonstructions just because it’s simplest to, but i’m sure the same problems apply to varying degrees to other fontstructions.
    if you compare the ratings given to Structurosa to those given to K FontStruct (the top 2 fontstructions according to balanced ratings) it is true that K Fontstruct has 90% of votes as 10 and one only 3 votes are 1 and one of those is mine. I believe that the 101 votes of 10 are due to positrolls (although i can’t prove it) as it is curious that there are no votes from the range of 2-9. That seems very fishy to me.
    nonetheless, there are other factors that should perhaps contribute to greater weighting. although stucturosa has a lower rating (balanced or unbalanced) there it has twice as many ratings of 10 than K Fontstruct. It also has a range of votes that shows a more typical distribution of votes. There should be some mechanism that take into account the number of positive votes and add the appropriate amount of weight for more positive votes.
    another aspect that a ‘balanced’ rating scheme should take into account is downloads. although some fontstruct users may not vote using the rating system, they may certainly vote with downloads. again comparing structurosa and K Fontstruct you can notice something fishy when K Fontstruct has a higher rating with less than 20 downloads and structurosa has had 6250+ downloads. perhaps you don’t want to conflate download numbers with ratings, but i think that a true ‘balanced rating’ would take this into account.
    there will always be some distortions, i’m not sure the best way to iron them out. i applaud you for attempting to do so. one other idea that might help perhaps would be giving different voters votes different weights. so perhaps a voter who has only voted once will have a lower voting weight than someone who has voted, say 100 times. also if someone only votes 1′s or 10′s an algorithm could give these voters ratings a lower weight. just an idea… anyway, keep up the good work!

    Comment by paul d hunt — September 11, 2009 #


  14. Magic Sam

    @stewf: Hmm, maybe I´ll get used to it. But it seems that now most new Fontstructions will have a rating in the region of 6.0 – 7.0. I still like the old rating system more, but let´s wait & see.

    But please try to correct all the “0.0-balanced-rating”-errors.

    Comment by Magic Sam — September 11, 2009 #


  15. Gustavo Ferreira

    Thanks everyone for the comments — keep them coming! The balancing formula can be adjusted over time to produce better results. We’ll look into the ’0.0′ problem as soon we can. Have a good weekend!

    Comment by Gustavo Ferreira — September 11, 2009 #


  16. afrojet

    I must confess that I have been a lazy voter, favoring comments instead of votes. Nor have I paid much attention to the ratings system. But it seems that, in order for this to work, I need to be more proactive with my voting and understand the system better.

    With the new voting transparency, I did some number crunching on my top picked Fontstructions and found some interesting things. I have 41 shared top picks. Those 41 Fontstructions have garnered 663 total votes. Almost 1/5 of all votes cast for my TP’d Fontstructions, 18% (121 votes) have been “1″ votes. On the other hand, over 2/5, 41% have been “10″ votes. With eight of my Fontstructions the “1″ votes are equal to or greater than the “10″ votes.

    To me, these numbers indicate that we’re moving in the right direction by introducing a more balance ratings system that gives less power to the extreme ends of the bell curve.

    Comment by afrojet — September 11, 2009 #


  17. cayo

    I usually don’t rate either if I think my vote would downgrade the already given rating, but now I gave somebody a higher rating and it did downgrade it instead. I feel like an unintentional troll and it kind of makes me want to be careful about it.

    I just read Paul’s comment and it sounds well funded. Perhaps “Poor” Rob could apply a future fix so downloading and commenting (not made by ourselves, of course) would come to some value in the rating system. I cross my fingers on that one.

    Comment by cayo — September 11, 2009 #


  18. Rob Meek (meek)

    The zero rating thing should be fixed. A cache needed to be cleared ;-)

    Comment by Rob Meek (meek) — September 11, 2009 #


  19. Rob Meek (meek)

    @paul d hunt: yes there is something very fishy about the voting on those “K” fonts. I haven’t noticed that before (I don’t often look at that particular gallery view). I’ll take a look at that more closely when I can.

    Comment by Rob Meek (meek) — September 11, 2009 #


  20. SquarePeg

    Thank you for the detailed report and statistics. I agree with afrojet when he says “it’s a real eye opener to be able to drill down and look at those ratings graphs. Although I almost wish I had never seen ‘em – all those 1 votes – what a drag”…

    However, I want to give this modified system a chance. On the one hand, I too have been a lazy voter, and on the other, I’ve often cast compensatory “10″ votes to undo the actions of down-voting trolls.

    I like the Free Tags and Hidden Gems features; hopefully, they will help me become a better and more frequent voter.

    Thanks again for the improvements!

    Comment by SquarePeg — September 11, 2009 #


  21. paul d hunt

    just an evil little thought: trash all the 1-star votes and don’t tell anyone >^p

    Comment by paul d hunt — September 11, 2009 #


  22. paul d hunt

    oh and one more serious thought. i’d like to see a separate sort by the mean score and then by # of votes (or perhaps by # of downloads). just a thought.

    Comment by paul d hunt — September 11, 2009 #


  23. paul d hunt

    or maybe even by mean score and then balanced rating… okay i’ll stop now.

    Comment by paul d hunt — September 11, 2009 #


  24. Frodo7

    How can I post here anything that appears instantly, so other can read it?

    Comment by Frodo7 — September 11, 2009 #


  25. Frodo7

    First of all, I would like to thank Rob Meek his tireless effort to improve Fontstruct and its rating system. It is not an easy job, and I’m not talking about the technical aspects. Flex development seems to be the easy part. Without making any judgement, let me highlight a few thoughts about the topic.

    1. Voting and ranking is a sensitive issue. It affects people’s self-esteem, ambition, and vanity. You’ll never find a system that pleases everyone.

    2. “Lies, big lies and statistics.” FS staff still has the greatest leverage to (let’s put it gently) “influence” the lame voter. As your own data have clearly indicated that “Featured FontStructions get more and higher votes than Top Picks, which get more and higher votes than regular public FontStructions”. More exposure, more and higher votes. It’s as simple as that.

    3a. Balanced rating: I understand the sound principle behind the Bayesian ratings. A single or few 10s should not count as much as 50 votes with an average of 9.5. Clear as day. It was already clear for most of us without the Balanced rating. And I suspect you have been using it for a long time to rank fontstructions. How else could you rank them, anyway? But, it would have been better if you have kept this in the background. Now we’ve got two numbers (the balanced rating and the old average rating), that have virtually no relations to each other.
    3b. Furthermore, the balanced rating gets closer and closer to the true average as the number of votes increases, and gets reasonably close at a very high number of votes. But the overwhelming majority may never accrue that many. Thus, they left with a numerical value that vaguely tells something about the appreciation of their work, and as an indicator looks crude and tampered.

    Comment by Frodo7 — September 11, 2009 #


  26. Frodo7

    4. I was fascinated by your detailed account on the trolls. I didn’t know they were so diverse and fast evolving species. You have invalidated the votes of “Naughty Positrolls” but left the votes of “Down-voting trolls” (including “mega-trolls with 1000+ votes, average <2) untouched. This hardly points toward an act of balance.

    Comment by Frodo7 — September 11, 2009 #


  27. Frodo7

    5. Vote indicator and Hidden gems: useful features to increase the overall voting activity.

    It is very difficult to evaluate all the facets of a given fontstruction and squeeze it in a
    single or double digit. I am a perfectionist, obsessed by quality and detail. It is too easy to find some imperfections in most people’s work. There are only a handful truly impeccable designs. The majority of fontstructions, I’m sorry to say, deserve 5 or less. However, I respect the creative aspirations of others, and try to be generous. If I see a sparkle of creativity, I give it 10. Even if a work has apparent shortcomings, barking errors, I would
    rather not rate it, than discourage a budding talent. Call me Robin Hood, but I stick to this formula until a more sophisticated system is introduced.

    Comment by Frodo7 — September 11, 2009 #


  28. Gustavo Ferreira

    @frodo7: Comments on the blog are moderated (because of spam). Thanks a lot for your detailed review! I’ll reply in detail during the coming week.

    Comment by Gustavo Ferreira — September 12, 2009 #


  29. Frodo7

    6. The more votes, the better – I agree with this principle, and any measure that increase the voters turn up is positively welcome. However, we also know (at least since the Bush vs. Al Gore case, 2000) that many people simply do not vote. They walk into the boot and cast an empty ballot. Or they just stay at home. (A clear case for FS “under-vote”: when someone leaves a comment but gives not rating for reasons told or untold.) An
    under-vote or a no-vote is very informative. It could tell that a) people just can’t make up their mind, because too little information is given about the artwork; b) they don’t know how to vote: no clear guidance was given as to what count as 1,2,3,…10. c) they don’t trust the system; d) they don’t belive their vote will count/matter; e) they think it’s too much fuss about nothing, etc. My point is that these abstainings constitute a large body of opinion, we only have to learn how to listen to.

    Sorry to bother you on this lovely day.

    Comment by Frodo7 — September 12, 2009 #


  30. Gustavo Ferreira

    @frodo7: You’re not bothering at all. It’s fascinating to see how much thought and time you guys dedicate to FontStruct. Keep the good feedback coming, we’re reading!

    Comment by Gustavo Ferreira — September 12, 2009 #


  31. shasta

    Phew… Why do the big revolutions always take place when you’re on holiday? Away for one week, and the world is different when you come back… OK, I exaggerate. ;) But first of all: Thanks FS-staff! I’m still stunned by how dedicated you still are to this project, and how much you listen to the people’s voice. Great work!

    First thought on the new system: Great that you’ve not thrown over the 1-to-10-system completely, but tried to adjust it through statistical measures. I was a bit afraid that you’d replace it with some sort of “like/don’t like” thing that wouldn’t have allowed any precise distinction but only ranking by sheer quantity of votes.

    The balanced rating system also looks like a nice idea at first, but I’m afraid it has exactly the top-of-the-ranking-cementing effect I’d have feared about a “like/don’t like” system: Top ranked fonts will constantly get loads of (good) votes just because they’re top ranked and thus exposed to everyone, which won’t leave a chance to new fontstructions that start at 5 now, maybe get a handfull of votes while they’re in the live feed and the “what’s new” section and then just get stuck because nobody sees them anymore.

    And then I again have to take up the cudgels on behalf of boring, but usable typefaces (sorry if I bore you with this crusade of mine.;P): Fancy and spectacular fontstructions tend to get a lot more votes, just because they’re fancy and spectacular, which makes it much easier for them to get close to the top than for a unspectacular but maybe way more usable and complete fonts. Don’t get me wrong: Of course I think that creativity and innovation should get rewarded, but so should usability and thorough work. I’m afraid the new system might even amplify the already existing bias in favour of the “fancy” stuff.

    So I guess this comes down to my coming out as a non-supporter of the “the more votes, the better the rating” principle, or at least as a supporter of a much weaker application of it. I just think even the top of the ranking gets boring if it turns into a never-changing Olympus of impregnable vote titans.

    But in the end I also have to admit that I’ve never been a very active voter, that recently I’ve practically stopped voting for good, and that I’ve also stopped caring about it for a good deal… Encouraging and constructive comments and the feeling that my work is appreciated just mean much more to me.

    Comment by shasta — September 12, 2009 #


  32. Rob Meek (meek)

    Enjoying the discussion. I noticed that the new balanced ratings were always rounded to whole numbers. This has now been corrected.

    @shasta I like the Olympian metaphor and I certainly share your feeling about the greater value of positive and constructive comments. No ratings system is going to be perfect. I hope we can improve it a bit, and maybe make it a more useful sort criteria. Encouraging more votes is not enough in itself. As I said in my quote in Gustavo’s article, we also need to provide more ways for visitors and FontStruct lovers to access and explore the designs – hidden gems, tags, user homepages, connecting to other online social networks, the live feed, the multiple featured fontstructions – these are all attempts to make a wider range of work accessible. Of course it’s an ongoing process with many more changes, hopefully improvements, to come.

    Comment by Rob Meek (meek) — September 12, 2009 #


  33. Rob Meek (meek)

    @Frodo7 On the naughty positrolls and why we invalidated only their votes: the naughty positrolls are multiple, obviously fake accounts created by users solely to boost only their own ratings. So we invalidated the votes from these accounts. We are not invalidating the votes of the real users who created these accounts or penalizing them in any other way.

    The mega-trolls on the other hand, are real people who, for whatever reason, usually vote very low. But they also give high votes, to a diverse range of other users. So there is some warmth in their hearts. People have the right to vote low or high, the only problem is when people try to get around the principle of one person=one vote.

    Comment by Rob Meek (meek) — September 12, 2009 #


  34. Em

    33 comments: I saw it coming… :)

    I realized that there were going to be some bugs to be fixed on the first day (the 0.0 rated fonts, the ratings rounded to the whole numbers, etc.) so I did not rush to express my opinion.

    I always thought that the number of votes should influence the overall rating: a font getting a hundred of 9 ratings is definitely more appreciated than one getting only five 10 rating. The new balancing formula takes this factor into accounts and I’m happy about it.

    The time and thoughts the FS staff reserved to this issue, and how they are still open to suggestions, is really worthy of acknowledgment, so I think is unfair to complain about this different representation (how Rob explained) of the ratings fontstructions get. Besides the old rating system is still available in the sorting options.

    @Frodo7: as concerning trolls, and related sub-species, Gustavo explained very well their behavior, and I overall share the staff attitude towards them. Users giving many 1 ratings, but at the same time many other ratings in the 1-10 range as well, should not be banned, because I think there might be different good reasons to this behavior, for example considering a whole category of fonts really overrated.

    @Shasta: the gallery-as-Olympus issue is a real one, and it can get quite boring to see always the same fonts at the top: but remember there are several options to sort the gallery, even using key words, and the Hidden Gems feature adds a new possibility.

    @Paul D Hunt: I considered for a second the number of downloads a good parameter to affect the overall rating, but unfortunately is so easy to cheat on it by downloading repeatedly your own fonts, that I think is better to be left out. Again, there is the number of Downloads sorting option for this purpose.

    @Gustavo, just a small suggestion when picking the Featured Downloads: you could consider also picking older fontstructions that never got the right consideration in the past; there is no reason they have to be picked only among latest shared fonts.

    Comment by Em — September 14, 2009 #


  35. Gustavo Ferreira

    I just re-read all 30+ comments… a lot of information to process. It’s very rewarding to see you all getting involved and giving such great feedback.

    The new rating system has been in use for just three days now – it takes some time to get used to it, and some more time to see the global effect on the FontStruct ‘ecosystem’. I suggest we give the current formula some time, and then try to answer some basic objective questions together:
    - Is the new balanced rating better than average rating?
    - If yes, can the current formula be improved?
    - How?

    We can also compare rating data before and after the changes to see the precise effects on voting behavior.

    Comment by Gustavo Ferreira — September 14, 2009 #


  36. Gustavo Ferreira

    @Em: Re: Picking older FontStructions for the Featured Download – That’s a great suggestion. The current set has two slightly older Top Picks (Frucktur and FS 25), but I agree it would be nice to go even further back.

    If anyone has a particular suggestion for Featured FontStruction or Top Pick, just send us an email.

    Comment by Gustavo Ferreira — September 14, 2009 #