Let me tell you a story about enhancing an existing product with Generative AI. It will be a cautionary one, so if you read carefully, you may avoid some (potentially expensive) future mistakes.
As you may have already read on this blog, I pay a lot of attention to curating the information I consume. It's not only about deliberate learning but also filtering news and journalistic commentary. I pay for the (high quality) feeds and carefully consider if I want any personalization (& what kind of it). I also check news outlets' affiliations, their long-term sociopolitic bias, and fact-checkers' opinions on them.
To be more specific:
For news, I rely on Ground News. For journalism, I either follow trustworthy (IMHO) media directly (Quillette, Tortoise Media, currently I'm testing The Free Press) or use curated aggregators of the best content - like Curio.
What is ... WAS Curio
The idea was super simple. Instead of having a plethora of standalone subscriptions for popular media (like Economist, Foreign Policy, WaPo, Financial Times, The Atlantic, MIT Tech Review, and many others), I can rely on the Curio team that handpicks the most interesting articles from every issue (of these renowned magazines/journals), records them as audio episodes and makes available on their (paid) platform. Just one subscription instead of many, but still with access to materials from many proven sources. Needless to say, they have fully legit deals with all the publishers, so there's no intellectual property infringement. One convenient way to think about it (in a slightly oversimplified way) is that Curio is a "Netflix for commentary journalism".
Why and how I've used Curio
Knowing all the biases and imperfections of the above-mentioned media outlets, I would not go for an individual subscription to any of them. I don't want a one-sided, heavily opinionated perspective from one source I don't fully trust, even if I think I can filter it reasonably well. I need the breadth to see & confront sometimes contradicting views or just look at the given problem from different angles - that's precisely what Curio offered (in 1.0).
My "user journey" in the app wasn't very complex. It was possible to browse articles by source (magazine/publisher), topic, "hotness", and some other criteria, but I only used the "What's new" dynamic list (it's self-explanatory, I presume). I inspected that feed (organized in reverse chronological order - the latest first) every three days or so to pick the headlines I liked and add them to my queue. Such cadence was pretty much perfect - not too few and not too many items to examine each time.
Listening to what I've put in the queue was done asynchronously on various occasions during the day: when I was taking a stroll, enjoying some personal time, etc. So, as you can imagine, my priority was to optimize the queue's depth - to ensure it never dries out completely, but also don't let the stuff rot there for too long.
And that's when Curio decided to "improve" the app by peppering it with Gen AI. What could go wrong?
Curio 2.0 (codename "Rio")
I could only speculate on the chain of thought that had led to this change, so let me focus on the outcomes only:
Curio has replaced all the existing UI with a chat window. It starts empty (ChatGPT-style), and you have a small text box to put your prompts in & interact with Curio's "AI". To guide the (potentially?) confused user, they've added some hints - the most popular phrases people use (or are expected to use by Curio staff) in prompts (like "Editor's Picks", "Best of Curio", or "Today's Headlines").
OK, so it seems my old workflow is gone for good, but maybe I could replace it with an equivalent - e.g., asking for today's headlines in the chat? Let's see ...
Well, first of all, I've quickly learned NOT to use my own prompts. The LLM used by the chatbot is very dumb (probably because of the cost). For instance, when I asked for the "most recent news and articles", it provided me with all the output it could find ABOUT news and articles ... Yikes.
It works better if you click the pre-made prompt suggestions I've mentioned before, but ...
The actual results are:
- Limited by default (8 entries)
- NOT actual articles but their concise aggregations - shortened amalgamates of 2+ articles on the same (more or less) topic from various media
- Fortunately, under each summary, there's a list of articles it's based upon - you can click on each article, listen to it, or (not in all cases) read it as well.
- Finally, there's also an option to create a few minutes-long audio episode based on all these articles; it takes only a few seconds, and you're explicitly told which part of the created episode is based on which article.
Sounds good, doesn't it?
Why Curio 2.0 is bad, very bad
Oh, where do I start?
While a chatbot may seem like a great idea for simple & clean user interface design, it's a common truth that different UI patterns work worse or better for solving particular problems. Chatbot is fine if your user is purely exploring (at a high degree of ambiguity) with probing/open questions. Or (on the contrary) if (s)he precisely knows what (s)he's looking for.
However, chatbots are not the optimal choice for browsing already-categorized and organized information. Why so? Their navigability (ease and speed of moving between categories and levels of abstraction) is bad & slow (e.g., moving backward, "scrolling" to the next "portion", zooming in to details and back). Chatbot form also sucks for visual learners (the majority of the population), primarily because of limited information bandwidth (just text, no space to use common information architecture patterns) & interaction options.
The idea of aggregating articles based on the topic/event is also a flop. It'd be brilliant for short news as these are based on universal facts (btw. that's what Ground News does), but commentary journalism is all about opinions: highly individual, fresh, and controversial ones that come from various angles, are based in different sociopolitical viewpoint contexts, etc. Merging them seems pointless.
Last but not least - the assumption that a busy reader will open the app and ask a question about the specific topic of his choice (e.g., "what's new about Mark Zuckerberg?") seems ... wrong. I want to "outsource" this decision (choice of viable topics) to the commentators - what did they decide is worth public attention today/this week (as these are primarily weekly magazines), so they've written about it? Is it "Mark Zuckerberg" because there was something significant he did or that was done around him? Or maybe it's something completely different - e.g., the latest report on EU competitiveness that I didn't even know existed. Curio 2.0 doesn't help me spot exciting topics I haven't expected (yes, there's a list of "trending topics" in the chat suggestions, but these are hot topics according to Curio editors or their algorithm, not myself).
Conclusion. And is it (even) fixable?
Generative AI is awesome. Period. And one of its top use cases is improving user interfaces. However, one should pay attention to where precisely and how it's applied. In the case of Curio:
- Chatbot should not replace browsing (of the article "catalog") - let people roam free! Let people learn that they went through 100% of what has been added since their last session.
- But there should be an (additional) option for lost/bored/uncertain users who'd like to "phrase" their interest in a descriptive way (you can think about it as a smarter equivalent of search).
- Gen AI could also be used to ask the user some questions after reading the article - to pinpoint their preferences/interests/feedback (for future recommendations/enhancements).
- Aggregating articles makes very little sense, but using text-to-audio models to increase the number of articles available in audio form is definitely a step in an interesting direction (I think they are doing that already - in the previous version, professional human lectors read all the articles).
- The clunky Gen AI experience is frustrating and made me question whether the amount of input data (sources - publishers and how many articles are collected from them) hasn't been reduced. I pretty much never got what I expected & too often, I had an impression of running in circles.
My wild guess (not confirmed by any statement from the Curio team) is that this change (Curio 2.0) was just one step in the planned long-term transition - towards a full Gen AI experience. The transition from a curated stream of articles created by human journalists from renowned media brands to a personalized stream of automatically generated (by Gen AI, based on processed news streams) commentaries/analyses. Such a solution would be much cheaper than paying for big brand content. It would also make the audience "happy" by providing them with opinions perfectly fine-tuned to the beliefs and biases of their information bubbles.
But pardon me - when I think about such a perspective, I'm not an enthusiast.