Add rationale, product considerations, compatibility

FIPS/fip-0045.md

@@ -1,7 +1,7 @@
 ---
 fip: 0045
 title: De-couple verified registry from markets
-author: Alex North (@anorth)
+author: Alex North (@anorth), Zenground0 (@zenground0)
 discussions-to: https://github.com/filecoin-project/FIPs/discussions/313
 status: Draft
 type: Technical
@@ -11,14 +11,9 @@ requires: FIP-0034
 ---
 
 TODO:
-- Spec receiver hook on Account, Multisig actors
 - Spec methods for sector migration on miner, market, and verifreg actors
 - Spec change to SectorOnChainInfo to distinguish migrated sectors
-- Discuss changes to deal packing
-- Discuss how market delegate means no external API changes
-- Discuss mechanisms for conditional allocations, negatively priced deals
 - Confirm policy for built-in market actor's default term maximum
-- Confirm policy for sector migration any extension to term maximum beyond the sector's lifespan
 - Market method for fetching allocation ids for deals (needed?)
 - Spec hook/method for a different client extending the claim by spending data cap (needed?).
 
@@ -123,7 +118,6 @@ Due to the implementation of term enforcement (details below), clients should le
 between the minimum and maximum term to make the allocation practical for a provider.
 
 **Parameters**
-
 - The minimum allowed value for `TermMinimum` is 6 months.
 - The maximum allowed value for `TermMaximum` is 5 years (can be increased by a future FIP).
 
@@ -136,7 +130,7 @@ struct State {
     // ... Existing verified registry state ...
     
     // Allocations indexed by client, then by ID.
-    Allocations: HAMT[Address]AMT[AllocationID]Allocation
+    Allocations: HAMT[Address]HAMT[AllocationID]Allocation
 	
     // Sequence number for allocation IDs.
     NextAllocationId: AllocationID // uint64
@@ -204,7 +198,7 @@ struct State {
     // ... Existing verified registry state ...
 
     // Claims by provider ID, then Claim ID.
-    Claims HAMT[Address]AMT[ClaimID]Claim
+    Claims HAMT[Address]HAMT[ClaimID]Claim
 }
 ```
 
@@ -315,6 +309,10 @@ This token delegation mechanism allows this proposal to be implemented without c
 clients' or providers' workflows for existing functionality.
 Parties will continue to interact with the built-in market actor and verified registry actor directly.
 
+#### Token receiver hook in account and multisig actors
+The built-in account and multisig actors are changed to implement a receiver hook method, as specified by FRC-0046.
+This hook always succeeds, without inspecting the payload.
+
 #### Operations
 The `AddVerifiedClient` method is changed to invoke the data cap token actor to
 mint new tokens to the client's balance.
@@ -455,7 +453,10 @@ When a deal proposal is published with `VerifiedDeal=true`, the market actor tra
 the corresponding data cap tokens from the client to the verified registry actor.
 This transfer is accompanied by an `Allocation` record populated with data from the deal.
 The built-in market specifies an allocation's minimum term to be equal to the deal duration,
-and maximum term to be one year greater than the deal duration.
+and maximum term to be some amount greater than the deal duration.
+
+**Parameters**
+- The built-in market calculates a term maximum of 1 year greater than a deal's specified duration.
 
 ```
 let dealAllocation = Allocation {
@@ -501,14 +502,13 @@ The miner actor then invokes `ClaimAllocations` and, if successful,
 computes quality adjusted power for the sector according to the piece size and QA power multiplier.
 
 ### Migration
-TODO!
-
 The power for existing sectors is not changed by this proposal.
 
-A provider may opt-in to migrate a sector to the new QA power calculation method.
+A provider may opt in to migrate a sector to the new QA power calculation method by invoking a new method.
 Each active deal is converted to a claim as if an allocation had been created when the deal was made.
-The claim's minimum term is the deal's duration, and maximum term is the sector's current expiration epoch
-(TODO: plus some buffer?).
+The claim's minimum term is the deal's duration, and maximum term is the greater of either
+the sector's current expiration epoch, or 540 days beyond the current epoch
+(since the provider could extend the sector to such lifetime prior to migrating).
 
 Expired deals cannot be migrated, even though the old QA power calculation method has spread out
 their power and reward over the sector's full lifetime.
@@ -527,68 +527,149 @@ After claims are extended, the provider can extend the sector commitment duratio
 Note that a migrated sector cannot be extended unless the claim terms are extended, 
 lest it violate a claim's maximum term.
 
-### Outline of main algorithms
-Describe the common flows.
-
+TODO: specify migration method and changes to SectorOnChainInfo
 
 ## Design Rationale
+The design rationale is mostly provided by the proposal's motivation.
+See also https://github.com/filecoin-project/FIPs/discussions/298.
 
-<!--The rationale fleshes out the specification by describing what motivated the design and why particular design decisions were made. It should describe alternate designs that were considered and related work, e.g. how the feature is supported in other languages. The rationale may also provide evidence of consensus within the community, and should discuss important objections or concerns raised during discussion.-->
+The rationale behind the new flows is to remove system trust in the built-in market actors.
+This proposal removes trust of the verified registry in the built-in market, 
+by moving network-critical state into the verified registry actor state directly.
 
-Remaining work to fully remove trust in market actor (API changes)
-
-Future capabilities
-- Internal transfer between sectors (requires new miner APIs)
-- Transferable claims to new provider (new APIs)
-
-See https://github.com/filecoin-project/FIPs/discussions/298.
+Data cap is presented as a fungible token in order to support greater programmability
+in the future, such as through automated notary actors.
+Many tools and libraries will emerge to support fungible tokens generally, 
+and conforming to the same standard will allow data cap to enjoy their transparent support.
 
 ## Backwards Compatibility
-<!--All FIPs that introduce backwards incompatibilities must include a section describing these incompatibilities and their severity. The FIP must explain how the author proposes to deal with these incompatibilities. FIP submissions without a sufficient backwards compatibility treatise may be rejected outright.-->
-TODO
+This proposal changes the behaviour and internal APIs of several built-in actors, and thus requires a network upgrade. 
+The proposal also changes state schema of several built-in actors, which requires a state migration at that upgrade.
+Off-chain tools that directly inspect actor state may need to change to handle the post-migration schema.
+
+This proposal leaves all existing externally-invoked actor methods intact and functional.
+This is acheived through the built-in market actor acting as a delegate for data cap, 
+without needing to be trusted by the verified registry.
+Deal clients and providers are not expected to need to change their operations.
+
+Sectors containing verified data claims that are either committed after this upgrade, 
+and sectors which are migrated to verified data claims,
+can only have their lifetimes extended via the new `ExtendSectorExpiration2` method specified by this proposal.
+Storage providers must invoke this new method in order to extend such sectors.
+Sectors with no verified claims, including existing sectors which are not migrated, 
+can continue to use the existing `ExtendSectorExpiration` method.
 
 ## Test Cases
-<!--Test cases for an implementation are mandatory for FIPs that are affecting consensus changes. Other FIPs can choose to include links to test cases if applicable.-->
-TODO
+To be provided with implementation, prior to acceptance.
 
 ## Security Considerations
 <!--All FIPs must contain a section that discusses the security implications/considerations relevant to the proposed change. Include information that might be important for security discussions, surfaces risks and can be used throughout the life cycle of the proposal. E.g. include security-relevant design decisions, concerns, important discussions, implementation-specific guidance and pitfalls, an outline of threats and risks and how they are being addressed. FIP submissions missing the "Security Considerations" section will be rejected. A FIP cannot proceed to status "Final" without a Security Considerations discussion deemed sufficient by the reviewers.-->
 TODO
 
+- Mention circuit breaker possibility
+
 ## Incentive Considerations
 <!--All FIPs must contain a section that discusses the incentive implications/considerations relative to the proposed change. Include information that might be important for incentive discussion. A discussion on how the proposed change will incentivize reliable and useful storage is required. FIP submissions missing the "Incentive Considerations" section will be rejected. An FIP cannot proceed to status "Final" without a Incentive Considerations discussion deemed sufficient by the reviewers.-->
 TODO
 
+- SP preference for longer term maximums, shorter terms may be harder to fill
+
 ## Product Considerations
-<!--All FIPs must contain a section that discusses the product implications/considerations relative to the proposed change. Include information that might be important for product discussion. A discussion on how the proposed change will enable better storage-related goods and services to be developed on Filecoin. FIP submissions missing the "Product Considerations" section will be rejected. An FIP cannot proceed to status "Final" without a Product Considerations discussion deemed sufficient by the reviewers.-->
+Most of the product considerations of this proposal are directed towards the future product possibilities
+toward which this proposal works.
 
-TODO:
-- make third-party markets possible (after subsequent changes)
-- allocations larger than sectors (and towards deals similar), scalable representation
-- potential for direct route to FIL+ allocations (realised in next set of changes)
+TODO: Future capabilities
+- Internal transfer between sectors (requires new miner APIs)
 
 
+### Preparing for new market actors
+This proposal breaks the trust relationship between the verified registry actor (a core network component),
+and the built-in market actor.
+This is a step towards removing all network trust in market actors, at which point we can 
+support user-programmed market-like actors on the FVM.
+Such markets could implement a wide range of marketplace models and rules, and be significantly more efficient.
+The remaining step toward this goal is to remove the miner actor's trust of the built-in market actor.
+
+After this trust is fully removed, the built-in market actor could transition to an alternative governance model,
+decoupled from network upgrades.
+
+### Bypassing markets
+After the built-in market actor is decoupled from the miner actor,
+a client will be able to allocate data cap to a piece of data directly, 
+without necessarily using the built-in or any market actor.
+Such an allocation would represent the simplest, unconditional deal for verified data possible.
+This direct route would likely be far more gas-efficient than the built-in market actor.
+
+### Markets as delegates
+The "market as datacap delegate" pattern enables market actors to implement arbitrary deal logic, 
+and make the allocation of FIL+ data cap conditional upon the provider's acceptance of deal terms.
+For example, a market could specify a negative deal price (i.e. the provider pays the client),
+and allocte the data cap only after the provider has escrowed the necessary payment.
+Or a market could require simultaneous acceptance of a retrieval assurance policy as part of deal terms,
+again withholding the datacap allocation until on-chain conditions are met.
+Filecoin Plus verified clients will be able to choose markets that implement policies they want,
+which may extend far beyond what's reasonable to implement in the built-in market actor.
+
+### Multi-sector allocations
+The verified registry actor and its allocations and claims are not coupled to sector size.
+A future iteration to this schema could support allocations which are larger than a single sector,
+and may be claimed by a collection of data commitments spread across multiple sectors.
+
+Such an allocation would be far more gas efficient than the multiple per-sector deals
+required for a large data set today.
+
+### Support for FIP-0036
+Take alone, FIP-0036 could introduce an unfortunate disadvantage to storage providers with many verified deals,
+because such deals cannot be extended by the built-in market in order to enjoy higher duration multipliers.
+
+This proposal introduces two possible mechanisms to extend these existing deals.
+If a storage provider migrates a sector with verified deals to the proposed verified allocations, then either:
+- the deal client can extend the claim maximum term up to 5 years since it was committed, at no additional data cap
+- a different verified client can extend any claim up to 5 years from the current epoch, by spending additional data cap
+
+These mechanisms could restore equality of access to duration multipliers for existing sectors,
+but depend on client assent and intervention.
+
+### Gas costs
 The pattern of proxying verified registry interactions through the built-in market actor will
-initially raise the gas cost of deals, but this is a transitional state:
+initially raise the gas cost of publishing deals, and of committing a sector with verified claims,
+but this is a transitional state.
+This proposal is a stepping stone toward much greater potential for gas efficiency after
+the final decoupling from the miner actor.
 
-This proposal is a stepping stone toward greater potential gas efficiency of deals.
-- For verified data where the client is offering no payment above the verified rewards, 
+- For simple, unconditional verified data allocations, with no additional payment, 
   no deal with a market will be necessary at all.
   A client need only make an allocation directly with the verified registry.
-  This allocation is much cheaper to make and maintain than a deal.
-- The proposed representations make possible data cap allocations larger than a single sector.
-  As well as improvements in utility, such allocations will be far more gas efficient than
+  This direct allocation will be much cheaper to make and maintain than a deal.
+- Upgrades to the claim schema could support data cap allocations larger than a single sector.
+  As well as improvements in utility, such allocations would be far more gas efficient than
   negotiating a separate deal for each sector-sized unit.
-- For deals where additional payment or terms are necessary, 
-  this decoupling opens the potential to develop alternative storage markets, 
-  including ones much more efficient than the built-in one.
-  We can expect deal costs to decrease over time.
+- Decoupling FIL+ from markets opens the potential to develop alternative storage markets, 
+  including those far more efficient than the built-in one.
+
+By opening the field of competition for market-like actors, we can expect deal costs to decrease over time,
+while the network enforces just enough state to administer the quality-adjusted power of verified data. 
+
+### Deal packing
+Storage providers choose how to arrange deals into sectors, subject to the constraints on size and start epoch.
+They may optimise for various goals, such as filling space, or space-time, or latency to commit a sector.
+This proposal introduces a new constraint of an allocation's maximum term, 
+and removes a possible optimisation target of space-time.
+
+Each allocation constrains the sector's commitment duration to lie between its minimum and maximum term.
+Two data cap allocations may only be sealed into the same sector if there is some overlap between
+the ranges specified by their minimum and maximum term.
+Storage providers' deal packing algorithms must be updated to respect these constraints.
+
+Prior to this change, colocating deals with very different durations is merely an inefficient use of space-time, 
+which diluted and delayed a provider's power and rewards.
+After this change, some allocation will be incompatible, but every compatible collection of allocations
+will then grant maximal per-byte quality-adjusted power for the sector's full lifetime.
+
 
 ## Implementation
-<!--The implementations must be completed before any core FIP is given status "Final", but it need not be completed before the FIP is accepted. While there is merit to the approach of reaching consensus on the specification and rationale before writing code, the principle of "rough consensus and running code" is still useful when it comes to resolving many discussions of API details.-->
 Implementation is in progress on the `decouple-fil+` branch of the built-in actors repository:
 https://github.com/filecoin-project/builtin-actors/tree/decouple-fil+.
 
-
 ## Copyright
 Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).