FIP-16: Longer Casts

[sanjayprabhu]

Title: Longer Casts
Type: Standards FIP
Authors: @sanjayprabhu, @varunsrin

Abstract

Introduce a new type of cast that can have up to 1,024 characters.

Problem

Users want to write more expressive posts but are limited to 320 bytes. This means that some interesting posts that could be written are cut short or not written at all.

The original intent of this limit was to increase the number of casts a user could store on the network, since we were concerned that storage was going to be a scarce resource. We’re less concerned about that now since we’ve had a year of observing storage patterns in the wild and believe we can support an increase.

Specification

1. Casts now include a type field on the CastAddBody
2. The type may be set to CAST type and a LONG_CAST

Messages with the long cast type may include 321 to 1024 bytes of text in the text field, while messages with the cast type may only include upto 320 bytes.

message CastAddBody {
  repeated string embeds_deprecated = 1; // URLs to be embedded in the cast
  repeated uint64 mentions = 2; // Fids mentioned in the cast
  oneof parent {
    CastId parent_cast_id = 3; // Parent cast of the cast
    string parent_url = 7; // Parent URL
  };
  string text = 4; // Text of the cast
  repeated uint32 mentions_positions = 5; // Positions of the mentions in the text
  repeated Embed embeds = 6; // URLs or cast ids to be embedded in the cast
  
  // New
  CastType type = 7;
}

// length validation for `text` field enforced depending on cast type
// 320 for CAST, 321 to 1024 for LONG_CAST
enum CastType {
  CAST = 0;
  LONG_CAST = 1;
}

This change will be simple for clients — they can extend their UI to allow more characters when typing and select the right type when publishing the data. They can also choose to display the entire text in the feed or truncate with a “Show more”.

The change is also simple for downstream recipients — they just need to update any database fields that store casts to support upto 1024 bytes.

Rationale

Why is the type necessary?

One approach would be to just increase the character count for all casts. The problem with this is that we might want different restrictions on long vs short casts later on. For example, long casts require more data and we might want to limit each storage unit to 100 long casts. This would be hard to do without the type.

Why not implement a new Message Type / Set?

Another approach would be to introduce a new set with its own limits. This creates backwards compatibility issues for all consumers of hub messages that must now understand a new type. It also requires expiration of long casts to be treated differently from short casts, which might not be necessary early on.

Won’t this increase storage on disk?

Currently there are 57M casts which consume 12GB or about 10% of disk space. The average size of a cast is 145 bytes (less than 100 chars) and the average user only has 165 casts. Assuming usage patterns continue to hold this should have minimal impact (1-2%) on storage.

What about storage consumption from adversarial actors?

It is quite possible that an adversarial account may buy storage and generate junk data to flood the hubs. In this case, they would be able to generate an additional 704 bytes per cast or 3MB per storage unit. If we observe this behavior we may update the network to add a limit of a certain number of long casts per storage unit.

[benadamsky]

Without digging too much into technical specs on how this affects hubs, I just wanted to briefly mention that segmenting long casts into its own bucket at the protocol layer seems like a promising solution. Just ~1k bytes provides a much more interesting surface area for specialized clients like Ponder in the immediate term without requiring the complexity of serving external cast data. Our team really appreciates all the effort going into long casts, and even better if it doesn't cause major disturbances to the existing cast structure for current clients 🫡

My only thoughts are around the rate of storage increase:

1. There might be some skewness in the average cast size due to replies (eg tipping) which isn't the most reliable indicator for a net new behavior like long casts, unless you're explicitly considering only parent casts.

2. On a more meta level, the introduction of long casts could potentially shift incentives towards posting longer form content over time, leading to a higher rate of storage increase than projected (eg once this is live we plan on promoting 100% long form over short form on our upcoming app). However, considering there's fixed storage on the long cast type, I'd imagine this will be a non-issue for a while, but still worth mentioning.

[sanjayprabhu]

Yes, storage is the main concern, we need to watch the usage patterns and decide next steps if it becomes a problem. Note that Casts and Long casts share the same storage right now (5000 shared between both), but we might need to separate and drastically reduce long casts (say 100-500 per unit) if it ends up being abused.

[vrypan]

How much additional storage is (1 x cast + 2 x replies) vs (1 x longer cast)?

I remember that at some point, it was mentioned that indexes require considerable storage, so maybe the overhead is not that much? And if this is the case, what's the length at which 1 long cast requires the same space as 3.2 casts? This may be a sweet spot (if it actually makes sense).

[devthakur9111]

Max time

[davidfurlong]

Sounds reasonable and simple - one uncertainty I have is what other CastTypes may exist in the future, and whether they will be in a disjoint union with CAST and LONG_CAST. Are there any ideas on what these might look like, and if none exist, whether a different abstraction naming might make sense (type feels general and ambigious).

The other uncertainty I have is whether 1k chars will satisfy the use case of people who want to write long casts. 3x longer isn't really that much longer, and certainly not long enough to write anything in-depth. Any data on this would be helpful

[vrypan]

I'd vote for CAST1024 (a la uint256). Most people don't care and will never know, but if you are the person to participate in these discussions, it's clear and it makes sense.

[sanjayprabhu]

Our convention is to just use type and have people infer it based on the containing object (see LinkBody, ReactionBody etc).

On the length, It's more of an educated guess with the caveat that we're open to changing the length if it's not sufficient. However, I don't think we'd increase it beyond, say, 2048 chars. The goal is to enable people to express slightly larger ideas (e.g. short screenshot essays) that don't fit in casts right now. And @vrypan's "article embed" FIP will cater to use cases where people want to share much longer form content.

I like CAST1024, but unfortunately, we can't do it for pragmatic reasons. We might change the length down the line, and this would ordinarily be fine if we only had GRPC APIs. But with http APIs, consumers are going to end up hardcoding CAST1024 as a string to check for this type, and it's going to cause confusion later.

[vrypan]

Do protobuf name changes actually have any impact on the implementation? I mean if CAST1024 was renamed CAST2048 in the protobuf description, would there be any implementation impact.

(Not that it's important, mostly out of curiosity)

[sanjayprabhu]

No impact. protobufs don't serialize the name over the wire, only the values (and not even that if it's the default)

[devthakur9111]

Waiting it but taking much time

[koisose]

@sanjayprabhu is there any example to use this with farcaster-js-hub-rest?