roost.tools / coop
Mirrored from GitHub github.com/roostorg/coop
0
Fork 0

Configure Feed

Select the types of activity you want to include in your feed.

ncmec: auto-populate priorCTReports, originalFileName, fileRelevance (#855)

* ncmec: auto-populate priorCTReports, originalFileName, fileRelevance

Closes three code-gap rows from the field-coverage audit on #843, all
auto-populated from data coop already has so no adopter configuration
is needed:

- `personOrUserReported.priorCTReports[]`: query
`ncmec_reporting.ncmec_reports` for accepted (`is_test=false`) report
IDs for the same reported user, ordered most-recent-first. Skipped for
exttest submissions since prod and test report IDs don't
cross-reference.
- `fileDetails.originalFileName`: derive from the media URL's pathname
(decoded last segment). Webhook-supplied `additionalInfo.fileName`
wins when present; URL-derived fills the gap for adopters without an
additional-info webhook.
- `fileDetails.fileRelevance`: default to `'Reported'` since
`submitReport`'s current call path only iterates over reviewer-flagged
media. Input slot accepts `'Supplemental Reported'` for future use.

Touch points:
- `NcmecReportingService.getPriorCTReportIds`: new query method.
- `BuildSubmitReportObjectInput.priorCTReports`: new input slot;
`buildSubmitReportObject` renders inside `personOrUserReported` after
`ipCaptureEvent` per XSD insertion order.
- `BuildFileDetailsObjectInput.{originalFileName,fileRelevance}`: new
input slots; `buildFileDetailsObject` renders them in their XSD
positions (3rd and 8th).
- `deriveOriginalFileNameFromUrl`: exported helper; `#upload` uses it
to populate `originalFileName` from `media.url` before calling
`buildFileDetailsObject`.

No migration; no GraphQL/SDL change; no client change. Purely additive
server behavior: existing reports gain three new elements, no existing
elements change.

Tests:
- `deriveOriginalFileNameFromUrl` (new describe block): decoding,
query/fragment stripping, unparseable URLs, malformed percent-encoding
fallback.
- `buildFileDetailsObject` (extended): updated minimum-envelope and
XSD-order expectations for the new elements; new cases for
`fileRelevance` default+override and `originalFileName` emit/omit.
- `buildSubmitReportObject` (new describe block): `priorCTReports`
populates in the right XSD position; omits when empty or unset.

All 69 NCMEC unit tests pass locally.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ncmec: log fallback paths in deriveOriginalFileNameFromUrl

Surfaces each of the four fallback cases (unparseable URL, empty
pathname, empty decoded segment, decode failure) via `ncmecDebugLog` so
operators can triage unexpected media URL shapes under `NCMEC_DEBUG=1`.
Addresses review nit on #855.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

author
juliet co-author
Claude Opus 4.7 (1M context) committer
GitHub date (Jun 29, 2026, 5:14 PM -0400) commit 6a482f1e parent 4a11305d
+198 -1
+44
server/services/ncmecService/buildSubmitReportObject.test.ts
··· 18 18 userAdditionalInfo?: BuildSubmitReportObjectInput['userAdditionalInfo']; 19 19 orgSettings?: Partial<BuildSubmitReportObjectInput['orgSettings']>; 20 20 clampedIncidentDateTime?: string; 21 + priorCTReports?: readonly number[]; 21 22 } = {}, 22 23 ): BuildSubmitReportObjectInput { 23 24 const { reportParams: paramOverrides, ...rest } = overrides; ··· 46 47 ...rest.orgSettings, 47 48 }, 48 49 clampedIncidentDateTime: rest.clampedIncidentDateTime ?? INCIDENT_DATE_TIME, 50 + ...(rest.priorCTReports !== undefined 51 + ? { priorCTReports: rest.priorCTReports } 52 + : {}), 49 53 }; 50 54 } 51 55 ··· 357 361 }), 358 362 ); 359 363 expect(result.report.reporter.termsOfService).toBe('Acme ToS'); 364 + }); 365 + }); 366 + 367 + describe('priorCTReports', () => { 368 + it('populates inside personOrUserReported after ipCaptureEvent', () => { 369 + const result = buildSubmitReportObject( 370 + makeBuildReportInput({ 371 + reportParams: { 372 + reportedUser: { 373 + id: 'user-1', 374 + typeId: 'user-type-1', 375 + ipAddress: '203.0.113.7', 376 + }, 377 + }, 378 + priorCTReports: [99, 102, 110], 379 + }), 380 + ); 381 + expect(result.report.personOrUserReported?.priorCTReports).toEqual([ 382 + 99, 102, 110, 383 + ]); 384 + expect(Object.keys(result.report.personOrUserReported!)).toEqual([ 385 + 'espIdentifier', 386 + 'espService', 387 + 'screenName', 388 + 'ipCaptureEvent', 389 + 'priorCTReports', 390 + ]); 391 + }); 392 + 393 + it('omits the element when empty or unset', () => { 394 + const unset = buildSubmitReportObject(makeBuildReportInput()); 395 + expect(unset.report.personOrUserReported).not.toHaveProperty( 396 + 'priorCTReports', 397 + ); 398 + const empty = buildSubmitReportObject( 399 + makeBuildReportInput({ priorCTReports: [] }), 400 + ); 401 + expect(empty.report.personOrUserReported).not.toHaveProperty( 402 + 'priorCTReports', 403 + ); 360 404 }); 361 405 }); 362 406 });
+64
server/services/ncmecService/ncmecReporting.builders.test.ts
··· 1 1 import { 2 2 buildFileDetailsObject, 3 + deriveOriginalFileNameFromUrl, 3 4 fileAnnotationArrayToNCMECFileAnnotation, 4 5 NCMECEvent, 5 6 NCMECFileAnnotation, ··· 7 8 8 9 const INCIDENT_DATE_TIME = '2026-05-27T18:00:00.000Z'; 9 10 11 + describe('deriveOriginalFileNameFromUrl', () => { 12 + it('returns the decoded last path segment', () => { 13 + expect( 14 + deriveOriginalFileNameFromUrl('https://cdn.example/a/b/cat.jpg'), 15 + ).toBe('cat.jpg'); 16 + expect( 17 + deriveOriginalFileNameFromUrl('https://cdn.example/a/my%20file.png'), 18 + ).toBe('my file.png'); 19 + }); 20 + 21 + it('ignores query strings and fragments', () => { 22 + expect( 23 + deriveOriginalFileNameFromUrl('https://cdn.example/img.jpg?token=abc#x'), 24 + ).toBe('img.jpg'); 25 + }); 26 + 27 + it('returns undefined for paths without a usable last segment', () => { 28 + expect( 29 + deriveOriginalFileNameFromUrl('https://cdn.example/'), 30 + ).toBeUndefined(); 31 + expect( 32 + deriveOriginalFileNameFromUrl('https://cdn.example'), 33 + ).toBeUndefined(); 34 + }); 35 + 36 + it('returns undefined for unparseable URLs', () => { 37 + expect(deriveOriginalFileNameFromUrl('not a url')).toBeUndefined(); 38 + expect(deriveOriginalFileNameFromUrl('')).toBeUndefined(); 39 + }); 40 + 41 + it('falls back to the raw segment on malformed percent-encoding', () => { 42 + expect( 43 + deriveOriginalFileNameFromUrl('https://cdn.example/a/%E0%A4.jpg'), 44 + ).toBe('%E0%A4.jpg'); 45 + }); 46 + }); 47 + 10 48 describe('fileAnnotationArrayToNCMECFileAnnotation', () => { 11 49 it('returns undefined for empty or missing input', () => { 12 50 expect(fileAnnotationArrayToNCMECFileAnnotation(undefined)).toBeUndefined(); ··· 59 97 fileId: 'ncmec-file-1', 60 98 fileViewedByEsp: true, 61 99 exifViewedByEsp: true, 100 + fileRelevance: 'Reported', 62 101 industryClassification: 'A1', 63 102 }, 64 103 }); ··· 71 110 // exttest rejects the report. 72 111 const result = buildFileDetailsObject({ 73 112 ...baseInput, 113 + originalFileName: 'photo.jpg', 74 114 media: { 75 115 industryClassification: 'A1' as const, 76 116 fileAnnotations: [NCMECFileAnnotation.GENERATIVE_AI], ··· 91 131 expect(Object.keys(result.fileDetails)).toEqual([ 92 132 'reportId', 93 133 'fileId', 134 + 'originalFileName', 94 135 'fileViewedByEsp', 95 136 'exifViewedByEsp', 96 137 'publiclyAvailable', 138 + 'fileRelevance', 97 139 'fileAnnotations', 98 140 'industryClassification', 99 141 'originalFileHash', ··· 121 163 expect(empty.fileDetails).not.toHaveProperty('originalFileHash'); 122 164 expect(buildFileDetailsObject(baseInput).fileDetails).not.toHaveProperty( 123 165 'originalFileHash', 166 + ); 167 + }); 168 + 169 + it('defaults fileRelevance to "Reported" and accepts an override', () => { 170 + expect(buildFileDetailsObject(baseInput).fileDetails.fileRelevance).toBe( 171 + 'Reported', 172 + ); 173 + expect( 174 + buildFileDetailsObject({ 175 + ...baseInput, 176 + fileRelevance: 'Supplemental Reported', 177 + }).fileDetails.fileRelevance, 178 + ).toBe('Supplemental Reported'); 179 + }); 180 + 181 + it('emits originalFileName when supplied, omits when not', () => { 182 + expect( 183 + buildFileDetailsObject({ ...baseInput, originalFileName: 'cat.jpg' }) 184 + .fileDetails.originalFileName, 185 + ).toBe('cat.jpg'); 186 + expect(buildFileDetailsObject(baseInput).fileDetails).not.toHaveProperty( 187 + 'originalFileName', 124 188 ); 125 189 }); 126 190
+90 -1
server/services/ncmecService/ncmecReporting.ts
··· 681 681 }; 682 682 /** Latest media `createdAt`, already clamped to the past. */ 683 683 clampedIncidentDateTime: string; 684 + /** Prior accepted NCMEC report IDs for the reported user; renders as `<priorCTReports>`. */ 685 + priorCTReports?: readonly number[]; 684 686 }; 685 687 686 688 /** Build the `Report` envelope NCMEC's `/submit` endpoint expects. Pure: no ··· 698 700 userAdditionalInfo, 699 701 orgSettings, 700 702 clampedIncidentDateTime, 703 + priorCTReports, 701 704 } = input; 702 705 const emailStringToNCMECEmail = (email: string) => ({ _text: email }); 703 706 ··· 814 817 reportedUserIpCaptureEvents.length > 0 815 818 ? { ipCaptureEvent: reportedUserIpCaptureEvents } 816 819 : {}), 820 + ...(priorCTReports && priorCTReports.length > 0 821 + ? { priorCTReports: [...priorCTReports] } 822 + : {}), 817 823 }, 818 824 ...(reportAdditionalInfo !== undefined 819 825 ? { additionalInfo: reportAdditionalInfo } ··· 825 831 export type BuildFileDetailsObjectInput = { 826 832 reportId: number; 827 833 fileId: string; 834 + /** Optional `<originalFileName>`. Usually derived via `deriveOriginalFileNameFromUrl`. */ 835 + originalFileName?: string; 836 + /** `<fileRelevance>`. Defaults to `'Reported'`. */ 837 + fileRelevance?: 'Reported' | 'Supplemental Reported'; 828 838 media: Pick<Media, 'industryClassification'> & { 829 839 fileAnnotations?: readonly NCMECFileAnnotationType[]; 830 840 }; ··· 838 848 originalFileHash?: readonly OriginalFileHash[]; 839 849 }; 840 850 851 + /** Decoded last path segment of `url`, or `undefined` if unavailable. 852 + * Logs each fallback path via `ncmecDebugLog` so operators can triage 853 + * unexpected URL shapes when `NCMEC_DEBUG=1` is enabled. */ 854 + export function deriveOriginalFileNameFromUrl(url: string): string | undefined { 855 + let pathname: string; 856 + try { 857 + pathname = new URL(url).pathname; 858 + } catch { 859 + ncmecDebugLog('deriveOriginalFileName.urlParseFailed', { url }); 860 + return undefined; 861 + } 862 + const last = pathname 863 + .split('/') 864 + .filter((s) => s !== '') 865 + .pop(); 866 + if (!last) { 867 + ncmecDebugLog('deriveOriginalFileName.emptyPath', { url }); 868 + return undefined; 869 + } 870 + try { 871 + const decoded = decodeURIComponent(last); 872 + if (decoded.length === 0) { 873 + ncmecDebugLog('deriveOriginalFileName.emptySegment', { url }); 874 + return undefined; 875 + } 876 + return decoded; 877 + } catch { 878 + // Malformed percent-encoding: keep the raw segment. 879 + ncmecDebugLog('deriveOriginalFileName.decodeFailed', { url, raw: last }); 880 + return last; 881 + } 882 + } 883 + 841 884 /** Pure builder for the `FileDetails` envelope NCMEC's `/fileinfo` 842 885 * endpoint expects. Used by `submitReport`'s `#upload` step and by 843 886 * dry-run tooling that needs to serialize per-media XML without ··· 845 888 export function buildFileDetailsObject( 846 889 input: BuildFileDetailsObjectInput, 847 890 ): FileDetails { 848 - const { reportId, fileId, media, additionalInfo, originalFileHash } = input; 891 + const { 892 + reportId, 893 + fileId, 894 + media, 895 + additionalInfo, 896 + originalFileHash, 897 + originalFileName, 898 + } = input; 899 + const fileRelevance = input.fileRelevance ?? 'Reported'; 849 900 const fileAnnotations = fileAnnotationArrayToNCMECFileAnnotation( 850 901 media.fileAnnotations, 851 902 ); ··· 853 904 fileDetails: { 854 905 reportId, 855 906 fileId, 907 + ...(originalFileName ? { originalFileName } : {}), 856 908 fileViewedByEsp: true, 857 909 exifViewedByEsp: true, 858 910 ...(additionalInfo.publiclyAvailable !== undefined 859 911 ? { publiclyAvailable: additionalInfo.publiclyAvailable } 860 912 : {}), 913 + fileRelevance, 861 914 ...(fileAnnotations ? { fileAnnotations } : {}), 862 915 industryClassification: media.industryClassification, 863 916 ...(originalFileHash && originalFileHash.length > 0 ··· 1669 1722 return firstReport != null; 1670 1723 } 1671 1724 1725 + /** Prior accepted NCMEC report IDs for the user, most recent first. 1726 + * Non-numeric `report_id`s are skipped (XSD requires `xs:integer`). */ 1727 + async getPriorCTReportIds(params: { 1728 + orgId: string; 1729 + userId: string; 1730 + userItemTypeId: string; 1731 + }): Promise<number[]> { 1732 + const { orgId, userId, userItemTypeId } = params; 1733 + const rows = await this.pgQuery 1734 + .selectFrom('ncmec_reporting.ncmec_reports') 1735 + .select(['report_id']) 1736 + .where('org_id', '=', orgId) 1737 + .where('user_id', '=', userId) 1738 + .where('user_item_type_id', '=', userItemTypeId) 1739 + .where('is_test', '=', false) 1740 + .orderBy('created_at', 'desc') 1741 + .execute(); 1742 + return rows 1743 + .map((r) => parseInt(r.report_id, 10)) 1744 + .filter((n) => Number.isFinite(n)); 1745 + } 1746 + 1672 1747 async submitReport( 1673 1748 reportParams: NCMECReportParams, 1674 1749 isTest: boolean, ··· 1850 1925 ); 1851 1926 } 1852 1927 1928 + // Skip for test submissions: prod and exttest report IDs don't 1929 + // cross-reference. 1930 + const priorCTReports = isTest 1931 + ? [] 1932 + : await this.getPriorCTReportIds({ 1933 + orgId: reportParams.orgId, 1934 + userId: reportParams.reportedUser.id, 1935 + userItemTypeId: reportParams.reportedUser.typeId, 1936 + }); 1937 + 1853 1938 const report = buildSubmitReportObject({ 1854 1939 reportParams, 1855 1940 userAdditionalInfo, ··· 1867 1952 moreInfoUrl: ncmecConfig?.more_info_url, 1868 1953 }, 1869 1954 clampedIncidentDateTime, 1955 + priorCTReports, 1870 1956 }); 1871 1957 1872 1958 // For the five actions here ··· 2174 2260 hmaHashes: media.hashes, 2175 2261 webhookFileDetails: additionalInfo.fileDetails, 2176 2262 }); 2263 + const originalFileName = 2264 + additionalInfo.fileName ?? deriveOriginalFileNameFromUrl(media.url); 2177 2265 const fileDetailsObject = buildFileDetailsObject({ 2178 2266 reportId: parseInt(reportId), 2179 2267 fileId, 2268 + ...(originalFileName ? { originalFileName } : {}), 2180 2269 media, 2181 2270 additionalInfo, 2182 2271 ...(originalFileHash ? { originalFileHash } : {}),