Benature/omnivore
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Make test SSML shorter

Browse Source
This commit is contained in:
Hongbo Wu
2022-08-31 18:45:23 +08:00
parent be8b0e6eb7
commit 3a40dd7266
2 changed files with 0 additions and 47 deletions
Download Patch File Download Diff File Expand all files Collapse all files

44
packages/text-to-speech/data/ssml.xml
View File

@ -33,48 +33,4 @@
<p><bookmark mark="47" />
The reality, though, is that this is what Instagram is best at. When Mosseri said that Instagram was no longer a photo-sharing app — particularly a “square photo-sharing app” — he was not making a forward-looking pronouncement, but simply stating what has been true for many years now. More broadly, Instagram from the very beginning — including under former CEO Kevin Systrom — has been marked first and foremost by evolution.
</p>
<p><bookmark mark="48" />To put this in Lessin’s framework, Instagram started out as a utility for adding filters to photos put on other social networks, then it developed into a social network in its own right. What always made Instagram different than Facebook, though, is the fact that its content was default-public; this gave the space for the rise of brands, meme and highlight accounts, and the Instagram influencer. Sure, some number of people continued to use Instagram primarily as a social network, but Meta, more than anyone, had an understanding of how Instagram usage had evolved over time.</p>
<p><bookmark mark="51" />In other words, when Kylie Jenner <bookmark mark="52" />posts a petition<bookmark mark="51" /> demanding that Meta “Make Instagram Instagram again”, the honest answer is that changing Instagram is the most Instagram-like behavior possible.</p>
<p><bookmark mark="53" />Three Trends</p>
<p><bookmark mark="54" />Still, it’s understandable why Instagram did back off, at least for now: the company is attempting to navigate three distinct trends, all at the same time.</p>
<p><bookmark mark="55" />The first trend is the shift towards ever more immersive mediums. Facebook, for example, started with text but exploded with the addition of photos. Instagram started with photos and expanded into video. Gaming was the first to make this progression, and is well into the 3D era. The next step is full immersion — virtual reality — and while the format has yet to penetrate the mainstream this progression in mediums is perhaps the most obvious reason to be bullish about the possibility.</p>
<p><bookmark mark="58" />The second trend is the increase in artificial intelligence. I’m using the term colloquially to refer to the overall trend of computers getting smarter and more useful, even if those smarts are a function of simple algorithms, machine learning, or, perhaps someday, something approaching general intelligence. To go back to Facebook, the original site didn’t have any smarts at all: it was just a collection of profile pages. Twitter came along and had the timeline, but the only smarts there was the ability to read a time stamp: all of the content was presented in chronological order. What made Facebook’s News Feed work was the application of ranking: from the very beginning the company tried to present users the content from their network that it thought you might be most interested in, mostly using simple signals and weights. Over time this ranking algorithm has evolved into a machine-learning driven model that is constantly iterating based on every click and linger, but on the limited set of content constrained by who you follow. Recommendations is the step beyond ranking: now the pool is not who you follow but all of the content on the entire network; it is a computation challenge that is many orders of magnitude beyond mere ranking (and AI-created content another massive step-change beyond that).</p>
<p><bookmark mark="61" />The third trend is the change in interaction models from user-directed to computer-controlled. The first version of Facebook relied on users clicking on links to visit different profiles; the News Feed changed the interaction model to scrolling. Stories reduced that to tapping, and Reels/TikTok is about swiping. YouTube has gone further than anyone here: Autoplay simply plays the next video without any interaction required at all.</p>
<p><bookmark mark="64" />One of the reasons Instagram got itself in trouble over the last few months is by introducing changes along all of these vectors at the same time. The company introduced more video into the feed (Trend 1), increased the percentage of recommended posts (Trend 2), and rolled out a new version of the app that was effectively a re-skinned TikTok to a limited set of users (Trend 3). It stands to reason that the company would have been better off doing one at a time.</p>
<p><bookmark mark="65" />That, though, would only be a temporary solution: it seems likely that all of these trends are inextricably intertwined.</p>
<p><bookmark mark="66" />Medium, Computing, and Interaction Models</p>
<p><bookmark mark="67" />Start with medium: text is easy, which is why it was the original medium of the Internet; effectively anyone can create it. The first implication is that there is far more text on the Internet than anything else; it also follows that the amount of high quality text is correspondingly high as well (a small fraction of a large number is still very large). The second implication has to do with AI: it is easier to process and glean insight from text. Text, meanwhile, takes focus and the application of an acquired skill for humans to interpret, not dissimilar to the deliberate movement of a mouse to interact with a link.</p>
<p><bookmark mark="68" />Photos used to be more difficult: digital cameras came along around the same time as the web, but even then you needed to have a dedicated device, move those photos to your computer, then upload them to a network. What is striking about the impact of smartphones is that not only did they make the device used to take pictures the same device used to upload and consume them, but they actually made it easier to take a picture than to write text. Still, it took time for AI to catch up: at first photos were ranked using the metadata surrounding them; only over the last few years has it become possible for services to understand what the photo actually is. The most reliable indicator of quality — beyond a like — remains the photo that you stop at while scrolling.</p>
<p><bookmark mark="69" />The ease of making a video followed a similar path to photos, but more extreme: making and uploading your own videos before the smartphone was even more difficult than photos; today the mechanics are just as easy, and it’s arguably even easier to make something interesting, given the amount of information conveyed by a video relative to photos, much less a text. Still, videos require more of a commitment than text or photos, because consuming them takes time; this is where the user interaction layer really matters. Lessin again, in <bookmark mark="70" />another Twitter screenshot:</p>
<p><bookmark mark="74" />
I saw someone recently complaining that Facebook was recommending to them…a very crass but probably pretty hilarious video. Their indignant response [was that] “the ranking must be broken.” Here is the thing: the ranking probably isn’t broken. He probably would love that video, but the fact that in order to engage with it he would have to go proactively click makes him feel bad. He doesn’t want to see himself as the type of person that clicks on things like that, even if he would enjoy it.
<bookmark mark="75" /> This is the brilliance of Tiktok and Facebook/Instagram’s challenge: TikTok’s interface eliminates the key problem of what people want to view themselves as wanting to follow/see versus what they actually want to see…it isn’t really about some big algorithm upgrade, it is about relesing emotional inner tension for people who show up to be entertained.
</p>
<p><bookmark mark="76" />This is the same tension between stated and revealed preference that Facebook encountered so many years ago, and its exactly why I fully expect the company to, after this pullback, continue to push forward with all three of the Instagram changes it is exploring.</p>
<p><bookmark mark="77" />Instagram’s Risk</p>
<p><bookmark mark="78" />Still, there is considerably more risk this time around: when Facebook pushed forward with the News Feed it was the young upstart moving aside incumbents like MySpace; it’s not as if its userbase was going to go backwards. This case is the opposite: Instagram is clearly aping TikTok, which is the young upstart in the space. It’s possible its users decide that if they must experience TikTok, they might as well go for the genuine thing.</p>
<p><bookmark mark="79" />This also highlights why TikTok is a much more serious challenge than Snapchat was: in that case Instagram’s network was the sword used to cut Snapchat off at the knees. I wrote in <bookmark mark="80" />The Audacity of Copying Well:</p>
<p><bookmark mark="82" />
For all of Snapchat’s explosive growth, Instagram is still more than double the size, with far more penetration across multiple demographics and international users. Rather than launch a “Stories” app without the network that is the most fundamental feature of any app built on sharing, Facebook is leveraging one of their most valuable assets: Instagram’s 500 million users…Instagram and Facebook are smart enough to know that Instagram Stories are not going to displace Snapchat’s place in its users lives. What Instagram Stories can do, though, is remove the motivation for the hundreds of millions of users on Instagram to even give Snapchat a shot.
</p>
<p><bookmark mark="83" />Instagram has no such power over TikTok, beyond inertia; in fact, the competitive situation is the opposite: if the goal is not to rank content from your network, but to recommend videos from the best creators anywhere, then it follows that TikTok is in the stronger relative position. Indeed, this is why Mosseri spent so much time talking about “small creators” with Newton:</p>
<p><bookmark mark="85" />
I think one of the most important things is that we help new talent find an audience. I care a lot about large creators; I would like to do better than we have historically by smaller creators. I think we’ve done pretty well by large creators overall — I’m sure some people will disagree, but in general, that’s what the data suggests. I don’t think we’ve done nearly as well helping new talent break. And I think that’s super important. If we want to be a place where people push culture forward, to help realize the promise of the internet, which was to push power into the hands of more people, I think that we need to get better at that.
</p>
<p><bookmark mark="86" />There is the old Internet AMA question as to whether you would rather fight a horse-sized duck or 100 duck-sized horses. The analogy here is that in a world of ranking a horse-sized duck that everyone follows is valuable; in a world of recommendations 100 duck-sized horses are much more valuable, and Instagram is willing to sacrifice the former for the latter.</p>
<p><bookmark mark="87" />Meta’s Reward</p>
<p><bookmark mark="88" />The payoff, though, will not be “power” for these small creators: the implication of entertainment being dictated by recommendations and AI instead of reputation and ranking is that all of the power accrues to the platform doing the recommending. Indeed, this is where the potential reward comes in: this power isn’t only based on the sort of <bookmark mark="89" />Aggregator dynamics<bookmark mark="88" /> underpinning dominant platforms today, but also absolutely massive amounts of investment in the computing necessary to power the AI that makes all of this possible.</p>
<p><bookmark mark="90" />In fact, you can make the case that if Meta survives the TikTok challenge, it will be on its way to the sort of moat enjoyed by the likes of Apple, Amazon, Google, and Microsoft, all of which have real world aspects to their differentiation. There is lots of talk about the $10 billion the company is spending on the Metaverse, but that is R<bookmark mark="90" />&amp;<bookmark mark="90" />D; the more important number for this moat is the $30 billion this year in capital expditures, most of which is going to servers for AI. That AI is doing recommendations now, but Meta’s moat will only deepen if Lessin is right about a future where creators can be taken out of the equation entirely, in favor of artificially-generated content.</p>
<p><bookmark mark="91" />What is noteworty is that AI content will be an essential part of any sort of Metaverse future; I wrote earlier this year in <bookmark mark="92" />DALL-E, the Metaverse, and Zero Marginal Content:</p>
<p><bookmark mark="94" />
What is fascinating about DALL-E is that it points to a future where these three trends can be combined. DALL-E, at the end of the day, is ultimately a product of human-generated content, just like its GPT-3 cousin. The latter, of course, is about text, while DALL-E is about images. Notice, though, that progression from text to images; it follows that machine learning-generated video is next. This will likely take several years, of course; video is a much more difficult problem, and responsive 3D environments more difficult yet, but this is a path the industry has trod before:
<bookmark mark="96" />Game developers pushed the limits on text, then images, then video, then 3D
<bookmark mark="97" />Social media drives content creation costs to zero first on text, then images, then video
<bookmark mark="98" />Machine learning models can now create text and images for zero marginal cost
<bookmark mark="99" /> In the very long run this points to a metaverse vision that is much less deterministic than your typical video game, yet much richer than what is generated on social media. Imagine environments that are not drawn by artists but rather created by AI: this not only increases the possibilities, but crucially, decreases the costs.
</p>
<p><bookmark mark="100" />These AI challenges, I would add, apply to monetization as well: one of the outcomes of Apple’s App Tracking Transparency changes is that advertising needs to shift from a deterministic model to a probabilistic one; the companies with the most data and the greatest amount of computing resources are going to make that shift more quickly and effectively, and I expect Meta to be top of the list.</p>
<p><bookmark mark="101" />None of this matters, though, without engagement. Instagram is following the medium trend to video, and Meta’s resources give it the long-term advantage in AI; the big question is which service users choose to interact with. To put it another way, Facebook’s next two decades are coming into sharper focus than ever; it is how well it navigates the TikTok minefield over the next two years that will determine if that long-term vision becomes a reality.</p>
<p><bookmark mark="103" />I wrote a follow-up to this Article in <bookmark mark="104" />this Daily Update.</p>
<p><bookmark mark="106" />Post navigation</p>
</prosody></voice></speak>

3
packages/text-to-speech/src/textToSpeech.ts
View File

@ -56,9 +56,6 @@ export const synthesizeTextToSpeech = async (
synthesizer.synthesizing = function (s, e) {
// convert arrayBuffer to stream and write to stream
console.debug(
`(synthesizing): Audio length: ${e.result.audioData.byteLength}`
)
writeStream.write(Buffer.from(e.result.audioData))
}