Intent

The agenda ahead:

• AI detectors don’t work – what’s up with that?

• Snapchat is the stealthy grower in social

• US regulators think fake reviews need to go

• How to track severe weather near you

///

Was James Madison a time traveler, or are AI detectors just really bad?

Midjourney: a friendly robot teacher lecturing a class full of human students --ar 4:1 --v 5.1

AI writing detectors (GPTZero, ZeroGPT, OpenAI’s own Text Classifier) have been less consistent than expected as the rise of AI content has accelerated. There are a not-insignificant amount of false positives being reported by enthusiasts and researchers, including in academia, where professors have begun attempts to snuff out low-effort/no-effort assignments from students using ChatGPT in bad faith.

One of the most recent viral examples: leading detectors are nearly certain that The US Constitution was written by AI. So, why does this happen?

Why don’t they work?

Similar to the LLMs they aim to identify, AI detectors are trained on a massive catalog of both human and AI-generated text. Then, they seek out a variety of indicators in sample content. Leading models focus heavily on two key metrics: “perplexity” and “burstiness.”

Perplexity translates to how surprising the language is to the model based on its training – in other words, how probabilistic is the content/sentence based on what these detectors know about LLM functionality. Burstiness quantifies how often certain words appear in rapid succession throughout a piece of text, as well as variation of sentence length.

Why those metrics aren’t perfect: if the text is something that AI expects to see (say, “a bowl of cereal” vs. “a bowl of lava”), then the detector could find it more likely that a piece of text was generated by AI if not properly tuned. Burstiness has the same problem—plenty of human-written content has uniform sentence structure and repetition, which we also expect to see out of these AI models.

Takes from an ethical AI expert

The short of it: these products just haven’t hit their detection benchmarks yet. So why are they so widely adopted? We asked Julian, an ethical AI expert in his current role, what’s happening here.

“This reminds me of the COMPAS system that was making the rounds in the state courts a couple of years ago. Everyone wanted hard metrics for criminal justice sentencing so badly that they forgot to consider that there may be issues with the historical data those models were trained on. Organizations are quick to adopt these tools because they appear to solve complex and uncomfortable problems while relying on a third party to support decision-making.

There is often little demand for transparency and vendors are quick to fill these niches with proprietary algorithms. I think we'll see the adoption of similar tools in many more universities before the consequences are fully understood.”

What’s next?

We’re not sure. Popular plagiarism detector TurnItIn has launched an AI detection add-on with millions of instructors already using it. Is it actually helping solve the problem, or will it continue to send innocent students into full-blown panic attacks?

///

Snapchat reaches 4 million premium subscribers

Midjourney: abstract art, a group of social media influencers, but they're all wearing yellow snapchat shirts and on their phones --ar 4:1 --v 5.1

Just one year after launching Snapchat+, the company announced that they’ve reached 4 million paid subscribers. Snap’s premium service offers subscribers early feature releases, exclusive content, and access to experiments. Features include things like priority story replies, chat wallpapers, custom chat colors, and the all-important verified badge.

With just north of 350M monthly active users, that’s more than 1% of users who are paid subscribers. Compare that to Twitter Blue’s 0.12% of Twitter users, and Meta’s Verified program which is estimated to hit .03% of their total users by the end of the year.

With that in mind, it’ll be interesting to see if Snapchat’s stronger success out of the pay-for-play social media roadmap will serve as a guide to the other major players seeking to diversify beyond the advertising business model.

///

This week’s quick hits:

Netflix has quietly built up its gaming library. What does that mean for the future of the platform, and the future of streaming? — The Ringer

A deep dive into what makes Meta’s Llama 2 LLM so incredible — Interconnects

How Roku has positioned itself as the streaming platform for streaming platforms — How It Grows

As companies consolidate SaaS products, they’re using less than half of their license — Productiv

Oddity is tech’s first IPO after the so-called “nuclear winter”; here’s what could happen next — @jstoffer

///

US regulators are cracking down on fake online reviews

Midjourney: paying for online reviews amazon --ar 4:1 --v 5.1

Late last week, the FTC proposed new legislation that would ban businesses from paying for reviews, suppressing honest reviews, selling fake social media engagement, and more. And this might’ve been long overdue.

According to recent studies, up to 30% of online reviews are classified as fake (CITE), with an impact on more than $150B in global consumer spending every year. The proposed ruling would leverage “hefty penalties” against the so-called bad actors. Beyond that, Google and Amazon have shown their own desire to boost platform trust — both companies have sued large review farms saturating their platforms with bought-and-paid-for content.

So, will this proposal change anything? From a macro level, the reality of regulation seems difficult – there are over ~350 products on Amazon that would have to be monitored, whether by regulators, watchdogs, or Amazon itself.

Plus, Google has already made significant investments into autoregulating fake content – they have an extensive process humans in the loop with machine learning, even using pattern and mapping data to understand the validity of a potential review. Yelp has their own proprietary algorithms and datasets based around users’ longstanding travel & eating activity, and hefty penalties for those who break platform rules.

If there’s an easy or obvious solution to this one, we haven’t seen it yet.

///

Tech to help you stay prepared for severe weather

Midjourney: funny, artful, a cow being pulled around by a tornado with a chat bubble saying this is fine --ar 4:1 --v 5.1

The weather seems to keep surprising us here at our HQ in NYC – from thunderstorms, to flash floods, to wildfire smoke – and so we went on the hunt for some tech to keep us cool, dry, and safe, no matter where you might be.

• In case of emergency – The Red Cross has a pretty extensive and customizable app that allows you to prepare up to 40 different weather alerts and various step-by-step instructions in the case of an emergency.

• Don’t get stuck on E – GasBuddy – in case of evacuation, you want to make sure you can get your tank. GasBuddy is a reliable source for gas station locations, price maps, and outage tracking.

• Personal safety first – Android + iOS emergency functions are key – make sure you turn on the personal safety features on your phone. Here’s the run down for Android and iOS.

• Need a B2B solution? PredictHQ allows businesses to access multiple data streams through one API to stay ahead of severe weather.

• Just want weather? NOAA Weather Radar Live gives comprehensive radar views, Storm Shield provides extensive storm-based updates, and AccuWeather takes an all-around approach.

• For wildfire smoke – The AirNow app allows users to get the most up-to-date information on air quality levels, along with a fire and smoke map to see where the thickest layers of smoke have drifted. Plume Labs shares comprehensive maps with pollution hot spots and hourly air quality forecast as levels shift throughout the day.

• Want to be proactive? RiskFactor.com shows you all the events – from floods to fires – with an existing risk within your geography.

If you feel like nerding out, here’s some additional reading on what meteorologists wish you knew about weather apps.

///

Thanks for reading, let us know what you think!

(also, please note that unsubscribing here will also unsubscribe you from future NYC tech event emails)

///

Unsubscribe