TwitterDataMining

TwitterDataMining is best understood as a well-documented thesis rather than a software product, and that framing changes what you should expect from it. The write-up walks through the theory in unusual detail for a blog post: it explains LDA as a generative 'roll the dice' process, contrasts variational Bayes with Gibbs sampling, then explains precisely why Hoffman's Online VB is not truly online — it uses a static vocabulary and forgets old topics progressively slowly, so topic drift early in the stream poisons later results. That critique is the intellectual core of the project, and WOLDA answers it with a genuinely simple idea: keep only words that appear at least min_df times within the last L time slices, randomly initialize words that are new, and link consecutive time slices through a contribution factor C that is implemented by back-solving the equivalent OLDA update count. If you're a student or researcher studying streaming topic models under concept drift, this is one of the clearer walkthroughs you'll find, and the code is a working reference rather than pseudocode. The representative-tweet comparison is also useful: KL-mean, cosine distance, and maximum entropy are implemented side by side, which is exactly the kind of experiment a thesis should run and a production blog post usually skips. Sentiment analysis is the weaker half conceptually — SVM and logistic regression over n-grams plus five lexicons was reasonable in 2016 but is well behind current text classifiers, and the thesis itself frames it as a standard classification exercise. Preprocessing is where the real domain knowledge sits: CMU ArkTweetNLP POS tagging, collapsing three-or-more repeated letters, deleting URLs and @mentions, expanding negations like don't and can't correctly, and appending a NEG suffix to words between a negation and the next punctuation mark. Those details matter for anyone working with short, noisy text. Physics of the situation: the pipeline depends on the Twitter REST and Streaming APIs as they existed in 2016, the author states plainly that the APIs have changed since, and there's no changelog, release history, or maintenance signal of any kind. The visualization layer — treemaps, bubble charts, sunburst charts, hashtag statistics, geolocation heatmaps plus a Google Maps click-to-lat/long helper — is functional but built for demonstration, not for an analyst's daily workflow. There's no GUI in the packaged-product sense, no configuration file convention, and no support channel. Decide accordingly: treat it as courseware you fork and adapt, budget time for API migration, and don't put it in front of a stakeholder as a product.