Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CORL-2948] emit moderation actions #4377

Merged
merged 9 commits into from
Nov 9, 2023
10 changes: 7 additions & 3 deletions server/src/core/server/services/comments/pipeline/helpers.ts
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aggregating commentActions instead of actions. moderationAction is a single object rather than an array, and isn't aggregated because Moderation Actions set a status on the comment, and so short circuit the pipeline.

Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,16 @@ export function mergePhaseResult(
result: Partial<PhaseResult>,
final: Partial<PhaseResult>
) {
const { actions = [], tags = [], metadata = {} } = final;
const { commentActions = [], tags = [], metadata = {} } = final;

// If this result contained actions, then we should push it into the
// other actions.
if (result.actions) {
final.actions = [...actions, ...result.actions];
if (result.commentActions) {
final.commentActions = [...commentActions, ...result.commentActions];
}

if (result.moderationAction) {
final.moderationAction = result.moderationAction;
}

// If this result contained metadata, then we should merge it into the
Expand Down
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renaming field

Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ export const detectLinks: IntermediateModerationPhase = ({
// Add the flag related to Trust to the comment.
return {
status: GQLCOMMENT_STATUS.SYSTEM_WITHHELD,
actions: [
commentActions: [
{
actionType: ACTION_TYPE.FLAG,
reason: GQLCOMMENT_FLAG_REASON.COMMENT_DETECTED_LINKS,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -102,9 +102,10 @@ export interface ExternalModerationRequest {
tenantDomain: string;
}

// BOOKMARK: (marcushaddon) do we need a new type to maintain type safety with joi?
export type ExternalModerationResponse = Partial<
Pick<PhaseResult, "actions" | "status" | "tags">
>;
Pick<PhaseResult, "status" | "tags">
> & { actions: PhaseResult["commentActions"] };
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is because our public API for external moderation phases still accepts the actions field, but their type is now internally called commentActions


const ExternalModerationResponseSchema = Joi.object().keys({
actions: Joi.array().items(
Expand Down Expand Up @@ -267,6 +268,19 @@ async function processPhase(
return validateResponse(body);
}

/**
* Our external API still just has a concept of "actions", while
* internally we distinguish beteween "moderationActions" and "commentActions"
*/
const mapActions = (
response: ExternalModerationResponse
): Partial<PhaseResult> => {
return {
...response,
commentActions: response.actions,
};
};

export const external: IntermediateModerationPhase = async (ctx) => {
// Check to see if any custom moderation phases have been defined, if there is
// none, exit now.
Expand Down Expand Up @@ -317,8 +331,10 @@ export const external: IntermediateModerationPhase = async (ctx) => {
},
});

const mappedResponse = mapActions(response);

// Merge the results in. If we're finished, return now!
const finished = mergePhaseResult(response, result);
const finished = mergePhaseResult(mappedResponse, result);
if (finished) {
return result;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ export const recentCommentHistory = async ({
if (rate >= tenant.recentCommentHistory.triggerRejectionRate) {
return {
status: GQLCOMMENT_STATUS.SYSTEM_WITHHELD,
actions: [
commentActions: [
{
actionType: ACTION_TYPE.FLAG,
reason: GQLCOMMENT_FLAG_REASON.COMMENT_DETECTED_RECENT_HISTORY,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ export const repeatPost: IntermediateModerationPhase = async ({

return {
status: GQLCOMMENT_STATUS.SYSTEM_WITHHELD,
actions: [
commentActions: [
{
actionType: ACTION_TYPE.FLAG,
reason: GQLCOMMENT_FLAG_REASON.COMMENT_DETECTED_REPEAT_POST,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ export const spam: IntermediateModerationPhase = async ({

return {
status: GQLCOMMENT_STATUS.SYSTEM_WITHHELD,
actions: [
commentActions: [
{
actionType: ACTION_TYPE.FLAG,
reason: GQLCOMMENT_FLAG_REASON.COMMENT_DETECTED_SPAM,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ export const statusPreModerateNewCommenter = async ({

return {
status: GQLCOMMENT_STATUS.SYSTEM_WITHHELD,
actions: [
commentActions: [
{
actionType: ACTION_TYPE.FLAG,
reason: GQLCOMMENT_FLAG_REASON.COMMENT_DETECTED_NEW_COMMENTER,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,7 @@ export const toxic: IntermediateModerationPhase = async ({

return {
status: GQLCOMMENT_STATUS.SYSTEM_WITHHELD,
actions: [
commentActions: [
{
actionType: ACTION_TYPE.FLAG,
reason: GQLCOMMENT_FLAG_REASON.COMMENT_DETECTED_TOXIC,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,12 +31,10 @@ export const wordListPhase: IntermediateModerationPhase = async ({
if (banned.isMatched) {
return {
status: GQLCOMMENT_STATUS.REJECTED,
actions: [
{
actionType: ACTION_TYPE.FLAG,
reason: GQLCOMMENT_FLAG_REASON.COMMENT_DETECTED_BANNED_WORD,
},
],
moderationAction: {
status: GQLCOMMENT_STATUS.REJECTED,
moderatorID: null,
},
metadata: {
wordList: {
bannedWords: banned.matches,
Expand All @@ -45,7 +43,7 @@ export const wordListPhase: IntermediateModerationPhase = async ({
};
} else if (banned.timedOut) {
return {
actions: [
commentActions: [
{
actionType: ACTION_TYPE.FLAG,
reason: GQLCOMMENT_FLAG_REASON.COMMENT_DETECTED_BANNED_WORD,
Expand All @@ -68,7 +66,7 @@ export const wordListPhase: IntermediateModerationPhase = async ({
if (tenant.premoderateSuspectWords && suspect.isMatched) {
return {
status: GQLCOMMENT_STATUS.SYSTEM_WITHHELD,
actions: [
commentActions: [
{
actionType: ACTION_TYPE.FLAG,
reason: GQLCOMMENT_FLAG_REASON.COMMENT_DETECTED_SUSPECT_WORD,
Expand All @@ -82,7 +80,7 @@ export const wordListPhase: IntermediateModerationPhase = async ({
};
} else if (suspect.isMatched) {
return {
actions: [
commentActions: [
{
actionType: ACTION_TYPE.FLAG,
reason: GQLCOMMENT_FLAG_REASON.COMMENT_DETECTED_SUSPECT_WORD,
Expand All @@ -96,7 +94,7 @@ export const wordListPhase: IntermediateModerationPhase = async ({
};
} else if (suspect.timedOut) {
return {
actions: [
commentActions: [
{
actionType: ACTION_TYPE.FLAG,
reason: GQLCOMMENT_FLAG_REASON.COMMENT_DETECTED_SUSPECT_WORD,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ describe("compose", () => {
body: context.comment.body,
status,
metadata: {},
actions: [],
commentActions: [],
tags: [],
});
});
Expand All @@ -48,7 +48,7 @@ describe("compose", () => {
body: context.comment.body,
status,
metadata: { akismet: false, linkCount: 1 },
actions: [],
commentActions: [],
tags: [],
});
});
Expand All @@ -71,14 +71,14 @@ describe("compose", () => {

const enhanced = compose([
() => ({
actions: [flags[0]],
commentActions: [flags[0]],
}),
() => ({
status,
actions: [flags[1]],
commentActions: [flags[1]],
}),
() => ({
actions: [
commentActions: [
{
userID: null,
actionType: ACTION_TYPE.FLAG,
Expand All @@ -91,10 +91,10 @@ describe("compose", () => {
const final = await enhanced(context);

for (const flag of flags) {
expect(final.actions).toContainEqual(flag);
expect(final.commentActions).toContainEqual(flag);
}

expect(final.actions).not.toContainEqual({
expect(final.commentActions).not.toContainEqual({
body: context.comment.body,
actionType: ACTION_TYPE.FLAG,
reason: GQLCOMMENT_FLAG_REASON.COMMENT_DETECTED_LINKS,
Expand All @@ -111,7 +111,7 @@ describe("compose", () => {
body: context.comment.body,
status: GQLCOMMENT_STATUS.NONE,
metadata: { akismet: false },
actions: [],
commentActions: [],
tags: [],
});
});
Expand Down
19 changes: 15 additions & 4 deletions server/src/core/server/services/comments/pipeline/pipeline.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import { Config } from "coral-server/config";
import { MongoContext } from "coral-server/data/context";
import { Logger } from "coral-server/logger";
import { CreateActionInput } from "coral-server/models/action/comment";
import { CreateCommentModerationActionInput } from "coral-server/models/action/moderation/comment";
import {
Comment,
CreateCommentInput,
Expand All @@ -26,16 +27,26 @@ import { mergePhaseResult } from "./helpers";
import { moderationPhases } from "./phases";
import { WordListService } from "./phases/wordList/service";

export type ModerationAction = Omit<
export type CommentAction = Omit<
CreateActionInput,
"commentID" | "commentRevisionID" | "storyID" | "siteID" | "userID"
>;

export type ModerationAction = Omit<
CreateCommentModerationActionInput,
"commentID" | "commentRevisionID" | "storyID" | "siteID" | "userID"
>;

export interface PhaseResult {
/**
* actions are moderation actions that are added to the comment revision.
* moderationActions are moderation actions that are added to the comment revision.
*/
moderationAction?: ModerationAction;

/**
* commentActions are comment actions that are added to the comment revision.
*/
actions: ModerationAction[];
commentActions: CommentAction[];

/**
* status when provided decides and terminates the moderation process by
Expand Down Expand Up @@ -122,7 +133,7 @@ export const compose =
const final: PhaseResult = {
status: GQLCOMMENT_STATUS.NONE,
body: context.comment.body,
actions: [],
commentActions: [],
metadata: {
// Merge in the passed comment metadata.
...(context.comment.metadata || {}),
Expand Down
32 changes: 27 additions & 5 deletions server/src/core/server/stacks/createComment.ts
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ import {
import { ensureFeatureFlag, Tenant } from "coral-server/models/tenant";
import { User } from "coral-server/models/user";
import { isSiteBanned } from "coral-server/models/user/helpers";
import { removeTag } from "coral-server/services/comments";
import { moderate, removeTag } from "coral-server/services/comments";
import {
addCommentActions,
CreateAction,
Expand Down Expand Up @@ -347,11 +347,11 @@ export default async function create(
// is added, that it can already know that the comment is already in the
// queue.
let actionCounts: EncodedCommentActionCounts = {};
if (result.actions.length > 0) {
if (result.commentActions.length > 0) {
// Determine the unique actions, we will use this to compute the comment
// action counts. This should match what is added below.
actionCounts = encodeActionCounts(
...filterDuplicateActions(result.actions)
...filterDuplicateActions(result.commentActions)
);
}

Expand Down Expand Up @@ -424,13 +424,13 @@ export default async function create(
log.trace("pushed child comment id onto parent");
}

if (result.actions.length > 0) {
if (result.commentActions.length > 0) {
// Actually add the actions to the database. This will not interact with the
// counts at all.
await addCommentActions(
mongo,
tenant,
result.actions.map(
result.commentActions.map(
(action): CreateAction => ({
...action,
commentID: comment.id,
Expand All @@ -447,6 +447,28 @@ export default async function create(
);
}

if (result.moderationAction) {
// Actually add the actions to the database. This will not interact with the
// counts at all.
await moderate(
mongo,
redis,
config,
i18n,
tenant,
{
...result.moderationAction,
commentID: comment.id,
commentRevisionID: revision.id,
},
now,
false,
{
actionCounts,
}
);
}

// Update all the comment counts on stories and users.
const counts = await updateAllCommentCounts(mongo, redis, config, i18n, {
tenant,
Expand Down
Loading