Summary

Recently we introduced submission workflows. This allows collection managers to solicit submissions from users based on a prompt and provides methods for approving or declining spubs. What we did not include, however, was a clear review process. To solve this we need to address some issues with soliciting a review.

The first is, well, requesting a review. Who initiates this process? For spubs it may be the collection organizer or members of the collection. Or maybe the Author can just add a list of people they want as reviewers. For pubs it may be a friend or colleague. Or maybe you want to open the pub to multiple reviews. Where would a user even initiate a review if they aren’t explicitly seeking a manager of the scope a pub is in? Another issue is that reviews exist outside of the pub body. This means gathering reviews may prove confusing to what we can conceive of as ephemeral users, users who create traces or artifacts on the platform but do not have accounts(when I speak of users from here on out I mean these two classes). It could also prove confusing for reviewers, as they would have to navigate to a pub and open it in a new tab to view what they are also reviewing.

From this we know we need a clearer process for users to gather feedback and we need a way to organize and view those reviews. So, why not create methods for explicitly soliciting feedback and having the reviews live at the pub level? It would be nice if I could just send an email or link to a friend with some text like “hey, what do you think of this“.

note: are comments reviews? no. but I mean they are right there…

What is it?

Now that we’ve thought about that, I want to introduce a new way to do reviews. Authors will now be able to share a unique link with the potential reviewer. The Author will fill in some information(name, email) about the reviewer, and we will use that to generate a unique link for that user and pub. The user who receives the link will be asked to enter their email to verify. If the user is already a member of that community they will receive a message and/or with this link. After validation, the reviewer will be presented with a page that will look very similar to the published version of this page. Only an EditorObject will exist to the right of this text. This editor object may pop out to the size of the screen if the text becomes too long or the viewport shrinks on the desktop. In mobile this could be a modal that can be minimized whenever. There will be a submission button for the reviewer after filling out the text area. The unique link we created needs to have invalidation after some time period that puts the submitted review in an uneditable state. With this, we can consider reviews as long-form annotations of pubs.

Here are some additional motivations behind this thinking

(There is a world with another type of users - Reviewers - but that is for a future iteration of this feature when we will also include the ability to solicit and provide Reviews for submissions.)

Automatic reminders for reviewers (and admins!) -- deadline

Notifications

We may want to solicit reviews from multiple sources

Community review

It would also be nice to send a piece of text off with a request for review. So we have reviews and text-specific comments that get collected in something like this when reviews are populated.

Also, see:

Creative Destruction: The Structural Consequences of Scientific Curation

Wikipedia as a Review Model?

Both were found through a nice thread by Ethan Mollick

Consider Reviews as First Class Content

https://reviewsindh.pubpub.org/review-process

MVP

But to get there let’s solve the primary issues. That being sending a pub for review and collecting a review. This is just a cursory look at what we

Request for Review

In PubShareDialog a new component will render a ‘Copy Review URL‘ link. This component will be a button named ReviewButton . Once the user clicks this button a modal will pop up with a form asking for user Information. If a user does not exist a ReviewUser will be created, as well as a User derived from that ReviewUser (might want to seriously rethink this pattern tho). After this, a reviewHash will be generated from the user values. This hash will belong to the Review model. The reviewer field on the Review model gets a ReviewUser. The field status needs to be set The link generated can be copied and sent to a user. I do not know if notifications or emails will be MVP yet.

There will be a new route with the URI pattern: /review/{pubSlug}/{hash} . This page will only require view permissions. The route will render ReviewPub(?) . The props passed into Review Pub(🤔 ) will be the initialData , reviewData , and pubForReview(I assume that since this route operates with the minimal role canView the pub will render automatically with canView permissions but I’m probably wrong and need to handle that after getting scopeData and loginData).

There will need to be a validation of the loginData.user.id and loginData.user.email . Simple form input and buttons will be used to facilitate this. Failure means redirecting to the community page.

Success will mean the review pub will be created. A high-level (to remain flexible) overview is something like this:

const thing = () => <ReviewEditorObject style={align: ''} submissionButton={submissionButton()}/>

const ReviewPub = () => {
  <div>
    <Pub.ReadOnly.With(thing)/>
  <div/>
}

I need to do some coding to figure out how we want to render the pub being reviewed. We also want to anticipate buttons causing something else so it will be abstracted from ReviewEditorObject a bit. ReviewEditorObject will be the only place a user can update anything.

During onSubmit the Review.status is changed submitted

This review has been viewable in ReviewDashboard since the creation of a link. Members can respond to it as they did previous reviews. Only they are responding to reviews not author request prompts.

What is success?

1. Higher than average interaction rates on pre-published works — this would be reviews.

2. Users like the UI

3. Ben, Ian, Gabe, and Eric leave 20 comments about how this could be better

4. I know what a type is

Design Process

Feature 1

send a link with a message requesting a review to anyone, Review or ReviewUser(?) They will be prompted to leave a comment on a

Stories

1. As an author, I want to send a link to people from which a user can fill out a text box

2. As a reviewer, I will use this link to create reviews and persist them even if I am not a user

3. As a reviewer, I should not be able to see the reviews

Designs

TBD

Implementation Checklists

• How do people do this now and how does this change that? Reviews are not signposted well enough. The review dashboard is nice but it seems strange with how coupled request publication review is to release. Authors can now draft pubs and send them to anyone for review. This allows anyone to review a pub. All steps should be more clear and provide a review flow.

• What other parts of the system are impacted? The pub dashboard. Review prompts become actual pubs. In which case I would like to rethink how reviews are structured here. There will be a new button in the pub header but i’m not sure where

• Who has permission to do what? Authors create review links from their pub. Reviewers are only able to draft reviews and submit them.

• Mobile support? yes

• Active states? User info modal, user email verification, review pub textbody, the button handling the submission

• Error states? not being verified using email, not being able to submit review, not being able to create a link