Skip to main content

Reviews

Most people want reviews

Published onJun 09, 2022
Reviews
·

Summary

Recently we introduced submission workflows. This allows collection managers to solicit submissions from users based on a prompt and provides methods of approving or declining spubs. What we did not include, however, was a clear review process. To solve this we need to address some issues with soliciting feedback on a pub.

The first is, well, requesting a review. Who initiates this process? For spubs it may be the collection organizer or the members of the collection. For pubs it may be a friend or colleague. Or maybe you want to open the pub to multiple reviews. Where would a user even initiate a review if they aren’t explicitly a manager of the scope a pub is in? Another issue is that reviews exist outside of the pub body. This means gathering reviews may prove confusing to what we can conceive of as ephemeral users, users who can create traces or artifacts on the platform but do not have accounts(when I speak of users from here on out I mean these two classes). It could also prove confusing for reviewers, as they would have to navigate to a pub and open it in a new tab to view what they are also reviewing.

From this we know we need a clearer process for users to gather feedback and we need a way to organize and view those reviews. So, why not create methods for explicitly soliciting feedback and having the reviews live at the pub level? It would be nice if I could just send an email or link to a friend with some text like “hey, what do you think of this“.

note: are comments reviews? no. but I mean they are right there…

What is it?

As a blogger, I would like to send someone a request for review. That way I can solicit feedback without the need for capture. Very similar concept to Attribution and AttributionUser. (Though we may need a more robust way of representing ephemeral users.)

I’m not the only one, however:

(There is a world with another type of users - Reviewers - but that is for a future iteration of this feature when we will also include the ability to solicit and provide Reviews for submissions.)

Some things we may want to consider

On-platform users get a notification

Automatic reminders for reviewers (and admins!) -- deadline

Notifications

We may want to solicit reviews from multiple sources

Community review

It would also be nice to send a piece of text off with a request for review. So we have reviews and text-specific comments that get collected in something like this when reviews are populated.

Also, see:

Creative Destruction: The Structural Consequences of Scientific Curation

Wikipedia as a Review Model?

Both were found through a nice thread by Ethan Mollick

Consider Reviews as First Class Content

MVP

But to get there let’s solve the primary issues. That being sending a pub for review and collecting a review.

Request for Review

The PubShareDialog component will render a ‘Copy Review URL‘ link. The Pub model gets the string property reviewHash on it. This will be generated on pub creation like the view and edit hashes.

The renderHashRow function will generate a link for a pub with a view-only pub body(of the pub to be reviewed) and an editor in which a reviewer can submit a review. This editor object will exist as a DocJ node on the Review type. Similar to the abstract on the Submission model.

These reviews can be seen truncated in the reviews dashboard.

Not MVP

Request for comment - can send a link from the pop-up toolbar that requests comments on a selection of text

These comments get shown in a separate discussion thread that can be turned public or private in a side nav.

What is success?

  1. Higher than average interaction rates on pre-published works — this would be reviews.

  2. Users like the UI

  3. Ben, Ian, Gabe, and Eric leave 20 comments about how this could be better

  4. I know what a type is

Design Process

Feature 1

send a link with a message requesting a review to anyone, Review or ReviewUser(?) They will be prompted to leave a comment on a

Stories

  1. As an author, I want to send a link to people from which a user can fill out a text box

  2. As a reviewer, I will use this link to create reviews and persist them even if I am not a user

  3. As a reviewer, I should not be able to see the reviews

Designs

TBD

Implementation Checklists

  • How do people do this now and how does this change that? Reviews are not signposted well enough to be a coherent method for soliciting feedback from say, 20 colleagues. Instead, reviews will live at the pub level. This allows users to arbitrarily request feedback from anyone. And allows anyone to review a pub(???1).

  • What other parts of the system are impacted? The pub dashboard. Reviews become actual pubs. In which case I would like to rethink how reviews are structured here. But for now, the review tab should mirror the reviews happening in the review-pub.

  • Who has permission to do what? Authors create review links from their pub. Reviewers are only able to draft reviews and submit them.

  • Mobile support? Maybe?

  • Active states? There is a button link.

  • Error states? unique review pub URL per-user. Im thinking this is just a pub URL. It would be nice to arbitrarily declare a URL for some scope. We should render a review-only pub document.

Comments
6
Gabriel Stein:

Yeah. People always say they want both: pub-level and comments. For MVP, we probably only need pub-level.

But I love your thought of how to do that by soliciting review on text ranges.

And it would be nice to find a way to collate discussions made during a review session as part of that review, in the future.

Gabriel Stein:

love this description of it.

Gabriel Stein:

yeah, this sounds right to me. I envision the button generates an email that gets sent to either a pubpub user or an email address you enter, with a timeout. Similar to how submission accept/decline works now. That could give us the db structure to show review statuses — that is, if no one gets the link, we can show admins it timed out.

Gabriel Stein:

100%! My sense is we probably want a convenience function for admins to convert reviews into Pubs rather than by default, but idk!

Gabriel Stein:

I imagine combining this with with the text selection above. What if you can open a “review” and there’s one for the entire document AND any specific places the author/editor highlighted. Super cool.

Gabriel Stein:

Also, I’m not sure for an MVP of this you need to control this within received states in a spub. It would be cool to be able to do this from any pub, but build in a shortcut for it in the spub dash.

Gabriel Stein:

whoa, I never considered this, but it would be super, duper cool to be able to request a comment on a specific range of text! omg.

And/but, I wonder if it’s a bit of overkill for an MVP. I think what would satisfy most folks is just a single link to a pub that pulls up a basic feedback form for the whole thing.

Eric McDaniel:

Great idea!

I do agree that we should probably start with a general purpose feedback form. In a future iteration we could add a new workflow step where people answer feedback per-block in this exact manner.