Topics of 2009
December 29, 2009 at 2:11 pm #79453
Last year, someone did a super neat “almost-chart-thingy” (sorry, still working on my coffee) with all the different people arranged by the number of posts and replies. could someone do the same for topics, real or to which threads drifted? The discussion of Foer’s book that has come back to farming practices, peak oil, and I’m sure by EOD, streetcars, brought this to mind.December 29, 2009 at 6:52 pm #329298
I think you mean this? http://www.columbusunderground.com/forums/topic/cu-post-election-thread-visualizations
What I hear you specifically asking for is a rather interesting thing, which I take to mean an answer to a question like “who participated in a thread, and then what, over time, were the statistically significant words that people used in that thread?” … or at least that’s how I would frame it in a way a computer might be able to answer…. is that close to what you are looking for? Feel free to expound post-caffeine, it took me a little of that stuff too to think about your post better :PDecember 29, 2009 at 8:17 pm #329299
The “cloud image” yes yes yes! I was thinking key words more than participants, but that would be very cool, too. (caffeine doesn’t seem to make me much more eloquent than I am first thing in the AM)December 29, 2009 at 10:17 pm #329300
Name some threads, I’ll see what I can do… just that one? Or some others? Thanks! Could be a little fun thing to explore for sure.December 29, 2009 at 11:17 pm #329301
Walker EvansKeymasterLogin to Send PM
I’d love to just see some of the same updated visualizations for 2009 if you have the time, th0m. Fun stuff! :DDecember 30, 2009 at 1:25 pm #329302
I was mostly wondering how many threads ended up being about or containing a lot of posts about farming practices.December 30, 2009 at 9:20 pm #329303
Berdawn — I need to download the full text of all of the threads, and I’m working on that, but I did an initial analysis of just the names of threads from 2009, and the result is this:
here’s a pdf: http://bit.ly/4vhxuY
here’s a (slightly hand-cleaned) word list: http://bit.ly/4u9B2L
here’s a list of urls,titles,viewcount,pagecount: http://bit.ly/8djwmK
here’s the python: https://gist.github.com/ccc95caab384e2b41525
So that is just based on words in topic names for the year, so probably not too surprising. Now that I have this, I can go back and get the actual threads’ content, and start doing some stats. Although someone actually talking about farming, or me here talking about how someone might want to find out about people talking about farming, plus you stating that’s what you are looking for, might all appear to be the same thing … when you break things into just words and do stats on the word occurrence. I may be able to try and do phrasing or something, I’ll keep digging and see what I can find.December 30, 2009 at 9:30 pm #329304
catnfiddleParticipantLogin to Send PM
Oooo, that’s nifty! Could be used as a banner graphic.December 30, 2009 at 10:10 pm #329305
BearParticipantLogin to Send PM
catnfiddle wrote >>
Oooo, that’s nifty! Could be used as a banner graphic.
Actually, if you could link from each of those words to a dynamically-generated list of threads whose titles contain it, it could be an attractive and useful CU banner.December 30, 2009 at 10:49 pm #329306
a tag cloud was a big thing in mybe 2007 and earlier and a lot of sites still have them, but mostly the problem with them is that over time they don’t change much if you base them on just popularity, they can be ‘spammed,’ they can get weighted in ways that aren’t very useful maybe, and some designers just think of them as random word vomits. :P
I am intrigued by berdawn’s orginal task, and that gets to more about what the topics were actually about more than just the title, but with the nice new change of meaningful and friendly urls for topics, i have to start with data like this, so it was a fun thing to do real quick and like the last post like it.
for the original questioni’d like to try an fit in a ‘sankey’ diagram but we’ll see what i find. maybe a zooming around video or something heheh but i can’t promise what will actually happen, just stating the kinds of things i wish i had more excuses to do :PDecember 30, 2009 at 11:01 pm #329307
Walker EvansKeymasterLogin to Send PM
Bear wrote >>
Actually, if you could link from each of those words to a dynamically-generated list of threads whose titles contain it, it could be an attractive and useful CU banner.
We do have something similar… a page that contains the most-used tags for messageboard posts:
They’re in alphabetical order to make them easy to find, and the size of the word displays the frequency in which it is used.
The “Popular Messageboard Topics” list in the right-column between the sidebar banner ads is actually a tag cloud with the size variation disabled, so those just the most used tags. The “more…” link at the end goes to the aforementioned page with loads more tags.
Useful? Perhaps. I don’t use it much myself.December 31, 2009 at 5:14 pm #329308
I messed with this a bunch last night, and apparently I only downloaded the most general of the general discussion section, so I will working on getting the rest. However, just looking at that there are something like 90 threads that eventually mention in one way or another something about farming, which is rather surprising (but then again I hadn’t been paying attention to these issues).December 31, 2009 at 6:22 pm #329309
and now this one! I’m going to see if I can’t come up with a reason to mention agribusiness and Rush Limbaugh.January 7, 2010 at 5:41 am #329310
Here are the top 15 users of the word “farming” and how many posts each user made that used it:
I am still messing around with all of this :PJanuary 7, 2010 at 12:34 pm #329311
ManateeMemberLogin to Send PM
Haha! Not surprising. I have a boner for farming ;P