Skip to content

fix: properly validate link content in AI JSON manipulation #1985

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

fix: properly validate link content in AI JSON manipulation #1985

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
5ebe73e
fix: properly validate link content in AI JSON manipulation
Hector-Zhuang Sep 5, 2025
f08fb8a
test: add JSON format test case for adding a link by AI
Hector-Zhuang Sep 5, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions ...24-08-06 (non-streaming)/add a link to google.com_1_82b3f34219ff0a515d07ae5202044dc3.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
{
"request": {
"method": "POST",
"url": "https://api.openai.com/v1/chat/completions",
"body": "{\"model\":\"gpt-4o-2024-08-06\",\"temperature\":0,\"messages\":[{\"role\":\"system\",\"content\":\"You're manipulating a text document using JSON blocks. \\n Make sure to follow the json schema provided. When referencing ids they MUST be EXACTLY the same (including the trailing $). \\n This is the document as an array of JSON blocks (the cursor is BETWEEN two blocks as indicated by cursor: true):\"},{\"role\":\"system\",\"content\":\"[{\\\"id\\\":\\\"ref1$\\\",\\\"block\\\":{\\\"id\\\":\\\"ref1\\\",\\\"type\\\":\\\"paragraph\\\",\\\"props\\\":{\\\"textColor\\\":\\\"default\\\",\\\"backgroundColor\\\":\\\"default\\\",\\\"textAlignment\\\":\\\"left\\\"},\\\"content\\\":[{\\\"type\\\":\\\"text\\\",\\\"text\\\":\\\"Hello, world!\\\",\\\"styles\\\":{}}]}},{\\\"cursor\\\":true},{\\\"id\\\":\\\"ref2$\\\",\\\"block\\\":{\\\"id\\\":\\\"ref2\\\",\\\"type\\\":\\\"paragraph\\\",\\\"props\\\":{\\\"textColor\\\":\\\"default\\\",\\\"backgroundColor\\\":\\\"default\\\",\\\"textAlignment\\\":\\\"left\\\"},\\\"content\\\":[{\\\"type\\\":\\\"text\\\",\\\"text\\\":\\\"How are you?\\\",\\\"styles\\\":{}}]}}]\"},{\"role\":\"system\",\"content\":\"The user asks you to do the following:\"},{\"role\":\"system\",\"content\":\"First, determine what part of the document the user is talking about. You SHOULD probably take cursor info into account if needed.\\n EXAMPLE: if user says \\\"below\\\" (without pointing to a specific part of the document) he / she probably indicates the block(s) after the cursor. \\n EXAMPLE: If you want to insert content AT the cursor position (UNLESS indicated otherwise by the user), then you need `referenceId` to point to the block before the cursor with position `after` (or block below and `before`).\\n \\n Prefer updating existing blocks over removing and adding (but this also depends on the user's question).\"},{\"role\":\"user\",\"content\":\"add a paragraph with the text 'Check this out' containing a link to 'https://www.google.com' after the last sentence\"}],\"tool_choice\":{\"type\":\"function\",\"function\":{\"name\":\"json\"}},\"tools\":[{\"type\":\"function\",\"function\":{\"name\":\"json\",\"description\":\"Respond with a JSON object.\",\"parameters\":{\"type\":\"object\",\"properties\":{\"operations\":{\"type\":\"array\",\"items\":{\"anyOf\":[{\"type\":\"object\",\"description\":\"Update a block, the new block will replace the existing block.\",\"properties\":{\"type\":{\"type\":\"string\",\"enum\":[\"update\"]},\"id\":{\"type\":\"string\",\"description\":\"id of block to update\"},\"block\":{\"$ref\":\"#/$defs/block\"}},\"required\":[\"type\",\"id\",\"block\"],\"additionalProperties\":false},{\"type\":\"object\",\"description\":\"Insert new blocks\",\"properties\":{\"type\":{\"type\":\"string\",\"enum\":[\"add\"]},\"referenceId\":{\"type\":\"string\",\"description\":\"MUST be an id of a block in the document\"},\"position\":{\"type\":\"string\",\"enum\":[\"before\",\"after\"],\"description\":\"`after` to add blocks AFTER (below) the block with `referenceId`, `before` to add the block BEFORE (above)\"},\"blocks\":{\"items\":{\"$ref\":\"#/$defs/block\"},\"type\":\"array\"}},\"required\":[\"type\",\"referenceId\",\"position\",\"blocks\"],\"additionalProperties\":false},{\"type\":\"object\",\"description\":\"Delete a block\",\"properties\":{\"type\":{\"type\":\"string\",\"enum\":[\"delete\"]},\"id\":{\"type\":\"string\",\"description\":\"id of block to delete\"}},\"required\":[\"type\",\"id\"],\"additionalProperties\":false}]}}},\"additionalProperties\":false,\"required\":[\"operations\"],\"$defs\":{\"styles\":{\"type\":\"object\",\"properties\":{\"bold\":{\"type\":\"boolean\"},\"italic\":{\"type\":\"boolean\"},\"underline\":{\"type\":\"boolean\"},\"strike\":{\"type\":\"boolean\"},\"code\":{\"type\":\"boolean\"},\"textColor\":{\"type\":\"string\"},\"backgroundColor\":{\"type\":\"string\"}},\"additionalProperties\":false},\"styledtext\":{\"type\":\"object\",\"properties\":{\"type\":{\"type\":\"string\",\"enum\":[\"text\"]},\"text\":{\"type\":\"string\"},\"styles\":{\"$ref\":\"#/$defs/styles\"}},\"additionalProperties\":false,\"required\":[\"type\",\"text\"]},\"inlinecontent\":{\"type\":\"array\",\"items\":{\"anyOf\":[{\"type\":\"object\",\"properties\":{\"type\":{\"type\":\"string\",\"enum\":[\"mention\"]},\"props\":{\"type\":\"object\",\"properties\":{\"user\":{\"type\":\"string\"}},\"additionalProperties\":false}},\"additionalProperties\":false,\"required\":[\"type\"]},{\"$ref\":\"#/$defs/styledtext\"},{\"type\":\"object\",\"properties\":{\"type\":{\"type\":\"string\",\"enum\":[\"link\"]},\"content\":{\"type\":\"array\",\"items\":{\"$ref\":\"#/$defs/styledtext\"}},\"href\":{\"type\":\"string\"}},\"additionalProperties\":false,\"required\":[\"type\",\"href\",\"content\"]}]}},\"block\":{\"anyOf\":[{\"type\":\"object\",\"properties\":{\"type\":{\"type\":\"string\",\"enum\":[\"paragraph\",\"quote\",\"toggleListItem\",\"bulletListItem\",\"numberedListItem\"]},\"content\":{\"$ref\":\"#/$defs/inlinecontent\"},\"props\":{\"type\":\"object\",\"properties\":{},\"additionalProperties\":false}},\"additionalProperties\":false,\"required\":[\"type\"]},{\"type\":\"object\",\"properties\":{\"type\":{\"type\":\"string\",\"enum\":[\"heading\"]},\"content\":{\"$ref\":\"#/$defs/inlinecontent\"},\"props\":{\"type\":\"object\",\"properties\":{\"level\":{\"type\":\"number\",\"enum\":[1,2,3,4,5,6]},\"isToggleable\":{\"type\":\"boolean\"}},\"additionalProperties\":false}},\"additionalProperties\":false,\"required\":[\"type\"]},{\"type\":\"object\",\"properties\":{\"type\":{\"type\":\"string\",\"enum\":[\"codeBlock\"]},\"content\":{\"$ref\":\"#/$defs/inlinecontent\"},\"props\":{\"type\":\"object\",\"properties\":{\"language\":{\"type\":\"string\"}},\"additionalProperties\":false}},\"additionalProperties\":false,\"required\":[\"type\"]},{\"type\":\"object\",\"properties\":{\"type\":{\"type\":\"string\",\"enum\":[\"checkListItem\"]},\"content\":{\"$ref\":\"#/$defs/inlinecontent\"},\"props\":{\"type\":\"object\",\"properties\":{\"checked\":{\"type\":\"boolean\"}},\"additionalProperties\":false}},\"additionalProperties\":false,\"required\":[\"type\"]},{\"type\":\"object\",\"properties\":{\"type\":{\"type\":\"string\",\"enum\":[\"table\"]},\"content\":{\"type\":\"object\",\"properties\":{}},\"props\":{\"type\":\"object\",\"properties\":{},\"additionalProperties\":false}},\"additionalProperties\":false,\"required\":[\"type\"]}]}}}}}]}",
"headers": [],
"cookies": []
},
"response": {
"status": 200,
"statusText": "",
"body": "{\n \"id\": \"chatcmpl-CCW1KZmIpbHqJEctRIaz5olIV7qtH\",\n \"object\": \"chat.completion\",\n \"created\": 1757099058,\n \"model\": \"gpt-4o-2024-08-06\",\n \"choices\": [\n {\n \"index\": 0,\n \"message\": {\n \"role\": \"assistant\",\n \"content\": null,\n \"tool_calls\": [\n {\n \"id\": \"call_W21fhJ2Dy9pJXFqBf49gV5Rg\",\n \"type\": \"function\",\n \"function\": {\n \"name\": \"json\",\n \"arguments\": \"{\\\"operations\\\":[{\\\"type\\\":\\\"add\\\",\\\"referenceId\\\":\\\"ref2$\\\",\\\"position\\\":\\\"after\\\",\\\"blocks\\\":[{\\\"type\\\":\\\"paragraph\\\",\\\"content\\\":[{\\\"type\\\":\\\"text\\\",\\\"text\\\":\\\"Check this out \\\",\\\"styles\\\":{}},{\\\"type\\\":\\\"link\\\",\\\"content\\\":[{\\\"type\\\":\\\"text\\\",\\\"text\\\":\\\"here\\\",\\\"styles\\\":{}}],\\\"href\\\":\\\"https://www.google.com\\\"}]}]}]}\"\n }\n }\n ],\n \"refusal\": null,\n \"annotations\": []\n },\n \"logprobs\": null,\n \"finish_reason\": \"stop\"\n }\n ],\n \"usage\": {\n \"prompt_tokens\": 1051,\n \"completion_tokens\": 74,\n \"total_tokens\": 1125,\n \"prompt_tokens_details\": {\n \"cached_tokens\": 1024,\n \"audio_tokens\": 0\n },\n \"completion_tokens_details\": {\n \"reasoning_tokens\": 0,\n \"audio_tokens\": 0,\n \"accepted_prediction_tokens\": 0,\n \"rejected_prediction_tokens\": 0\n }\n },\n \"service_tier\": \"default\",\n \"system_fingerprint\": \"fp_cbf1785567\"\n}\n",
"headers": []
}
}
15 changes: 15 additions & 0 deletions ...24-08-06 (non-streaming)/add a link to google.com_1_a07b53b9e3645cc50e882d6a23d17c0c.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
{
"request": {
"method": "POST",
"url": "https://api.openai.com/v1/chat/completions",
"body": "{\"model\":\"gpt-4o-2024-08-06\",\"temperature\":0,\"messages\":[{\"role\":\"system\",\"content\":\"You're manipulating a text document using JSON blocks. \\n Make sure to follow the json schema provided. When referencing ids they MUST be EXACTLY the same (including the trailing $). \\n This is the document as an array of JSON blocks (the cursor is BETWEEN two blocks as indicated by cursor: true):\"},{\"role\":\"system\",\"content\":\"[{\\\"id\\\":\\\"ref1$\\\",\\\"block\\\":{\\\"id\\\":\\\"ref1\\\",\\\"type\\\":\\\"paragraph\\\",\\\"props\\\":{\\\"textColor\\\":\\\"default\\\",\\\"backgroundColor\\\":\\\"default\\\",\\\"textAlignment\\\":\\\"left\\\"},\\\"content\\\":[{\\\"type\\\":\\\"text\\\",\\\"text\\\":\\\"Hello, world!\\\",\\\"styles\\\":{}}]}},{\\\"cursor\\\":true},{\\\"id\\\":\\\"ref2$\\\",\\\"block\\\":{\\\"id\\\":\\\"ref2\\\",\\\"type\\\":\\\"paragraph\\\",\\\"props\\\":{\\\"textColor\\\":\\\"default\\\",\\\"backgroundColor\\\":\\\"default\\\",\\\"textAlignment\\\":\\\"left\\\"},\\\"content\\\":[{\\\"type\\\":\\\"text\\\",\\\"text\\\":\\\"How are you?\\\",\\\"styles\\\":{}}]}}]\"},{\"role\":\"system\",\"content\":\"The user asks you to do the following:\"},{\"role\":\"system\",\"content\":\"First, determine what part of the document the user is talking about. You SHOULD probably take cursor info into account if needed.\\n EXAMPLE: if user says \\\"below\\\" (without pointing to a specific part of the document) he / she probably indicates the block(s) after the cursor. \\n EXAMPLE: If you want to insert content AT the cursor position (UNLESS indicated otherwise by the user), then you need `referenceId` to point to the block before the cursor with position `after` (or block below and `before`).\\n \\n Prefer updating existing blocks over removing and adding (but this also depends on the user's question).\"},{\"role\":\"user\",\"content\":\"add a paragraph with the text 'Check this out' and a link with text 'here' to 'https://www.google.com' after the last sentence\"}],\"tool_choice\":{\"type\":\"function\",\"function\":{\"name\":\"json\"}},\"tools\":[{\"type\":\"function\",\"function\":{\"name\":\"json\",\"description\":\"Respond with a JSON object.\",\"parameters\":{\"type\":\"object\",\"properties\":{\"operations\":{\"type\":\"array\",\"items\":{\"anyOf\":[{\"type\":\"object\",\"description\":\"Update a block, the new block will replace the existing block.\",\"properties\":{\"type\":{\"type\":\"string\",\"enum\":[\"update\"]},\"id\":{\"type\":\"string\",\"description\":\"id of block to update\"},\"block\":{\"$ref\":\"#/$defs/block\"}},\"required\":[\"type\",\"id\",\"block\"],\"additionalProperties\":false},{\"type\":\"object\",\"description\":\"Insert new blocks\",\"properties\":{\"type\":{\"type\":\"string\",\"enum\":[\"add\"]},\"referenceId\":{\"type\":\"string\",\"description\":\"MUST be an id of a block in the document\"},\"position\":{\"type\":\"string\",\"enum\":[\"before\",\"after\"],\"description\":\"`after` to add blocks AFTER (below) the block with `referenceId`, `before` to add the block BEFORE (above)\"},\"blocks\":{\"items\":{\"$ref\":\"#/$defs/block\"},\"type\":\"array\"}},\"required\":[\"type\",\"referenceId\",\"position\",\"blocks\"],\"additionalProperties\":false},{\"type\":\"object\",\"description\":\"Delete a block\",\"properties\":{\"type\":{\"type\":\"string\",\"enum\":[\"delete\"]},\"id\":{\"type\":\"string\",\"description\":\"id of block to delete\"}},\"required\":[\"type\",\"id\"],\"additionalProperties\":false}]}}},\"additionalProperties\":false,\"required\":[\"operations\"],\"$defs\":{\"styles\":{\"type\":\"object\",\"properties\":{\"bold\":{\"type\":\"boolean\"},\"italic\":{\"type\":\"boolean\"},\"underline\":{\"type\":\"boolean\"},\"strike\":{\"type\":\"boolean\"},\"code\":{\"type\":\"boolean\"},\"textColor\":{\"type\":\"string\"},\"backgroundColor\":{\"type\":\"string\"}},\"additionalProperties\":false},\"styledtext\":{\"type\":\"object\",\"properties\":{\"type\":{\"type\":\"string\",\"enum\":[\"text\"]},\"text\":{\"type\":\"string\"},\"styles\":{\"$ref\":\"#/$defs/styles\"}},\"additionalProperties\":false,\"required\":[\"type\",\"text\"]},\"inlinecontent\":{\"type\":\"array\",\"items\":{\"anyOf\":[{\"type\":\"object\",\"properties\":{\"type\":{\"type\":\"string\",\"enum\":[\"mention\"]},\"props\":{\"type\":\"object\",\"properties\":{\"user\":{\"type\":\"string\"}},\"additionalProperties\":false}},\"additionalProperties\":false,\"required\":[\"type\"]},{\"$ref\":\"#/$defs/styledtext\"},{\"type\":\"object\",\"properties\":{\"type\":{\"type\":\"string\",\"enum\":[\"link\"]},\"content\":{\"type\":\"array\",\"items\":{\"$ref\":\"#/$defs/styledtext\"}},\"href\":{\"type\":\"string\"}},\"additionalProperties\":false,\"required\":[\"type\",\"href\",\"content\"]}]}},\"block\":{\"anyOf\":[{\"type\":\"object\",\"properties\":{\"type\":{\"type\":\"string\",\"enum\":[\"paragraph\",\"quote\",\"toggleListItem\",\"bulletListItem\",\"numberedListItem\"]},\"content\":{\"$ref\":\"#/$defs/inlinecontent\"},\"props\":{\"type\":\"object\",\"properties\":{},\"additionalProperties\":false}},\"additionalProperties\":false,\"required\":[\"type\"]},{\"type\":\"object\",\"properties\":{\"type\":{\"type\":\"string\",\"enum\":[\"heading\"]},\"content\":{\"$ref\":\"#/$defs/inlinecontent\"},\"props\":{\"type\":\"object\",\"properties\":{\"level\":{\"type\":\"number\",\"enum\":[1,2,3,4,5,6]},\"isToggleable\":{\"type\":\"boolean\"}},\"additionalProperties\":false}},\"additionalProperties\":false,\"required\":[\"type\"]},{\"type\":\"object\",\"properties\":{\"type\":{\"type\":\"string\",\"enum\":[\"codeBlock\"]},\"content\":{\"$ref\":\"#/$defs/inlinecontent\"},\"props\":{\"type\":\"object\",\"properties\":{\"language\":{\"type\":\"string\"}},\"additionalProperties\":false}},\"additionalProperties\":false,\"required\":[\"type\"]},{\"type\":\"object\",\"properties\":{\"type\":{\"type\":\"string\",\"enum\":[\"checkListItem\"]},\"content\":{\"$ref\":\"#/$defs/inlinecontent\"},\"props\":{\"type\":\"object\",\"properties\":{\"checked\":{\"type\":\"boolean\"}},\"additionalProperties\":false}},\"additionalProperties\":false,\"required\":[\"type\"]},{\"type\":\"object\",\"properties\":{\"type\":{\"type\":\"string\",\"enum\":[\"table\"]},\"content\":{\"type\":\"object\",\"properties\":{}},\"props\":{\"type\":\"object\",\"properties\":{},\"additionalProperties\":false}},\"additionalProperties\":false,\"required\":[\"type\"]}]}}}}}]}",
"headers": [],
"cookies": []
},
"response": {
"status": 200,
"statusText": "",
"body": "{\n \"id\": \"chatcmpl-CCW3Hfm1wwzGAL0Br4JCj5X4f0enp\",\n \"object\": \"chat.completion\",\n \"created\": 1757099179,\n \"model\": \"gpt-4o-2024-08-06\",\n \"choices\": [\n {\n \"index\": 0,\n \"message\": {\n \"role\": \"assistant\",\n \"content\": null,\n \"tool_calls\": [\n {\n \"id\": \"call_hthJanljcECD3dXmhGssgY3D\",\n \"type\": \"function\",\n \"function\": {\n \"name\": \"json\",\n \"arguments\": \"{\\\"operations\\\":[{\\\"type\\\":\\\"add\\\",\\\"referenceId\\\":\\\"ref2$\\\",\\\"position\\\":\\\"after\\\",\\\"blocks\\\":[{\\\"type\\\":\\\"paragraph\\\",\\\"content\\\":[{\\\"type\\\":\\\"text\\\",\\\"text\\\":\\\"Check this out \\\"},{\\\"type\\\":\\\"link\\\",\\\"content\\\":[{\\\"type\\\":\\\"text\\\",\\\"text\\\":\\\"here\\\"}],\\\"href\\\":\\\"https://www.google.com\\\"}]}]}]}\"\n }\n }\n ],\n \"refusal\": null,\n \"annotations\": []\n },\n \"logprobs\": null,\n \"finish_reason\": \"stop\"\n }\n ],\n \"usage\": {\n \"prompt_tokens\": 1056,\n \"completion_tokens\": 67,\n \"total_tokens\": 1123,\n \"prompt_tokens_details\": {\n \"cached_tokens\": 1024,\n \"audio_tokens\": 0\n },\n \"completion_tokens_details\": {\n \"reasoning_tokens\": 0,\n \"audio_tokens\": 0,\n \"accepted_prediction_tokens\": 0,\n \"rejected_prediction_tokens\": 0\n }\n },\n \"service_tier\": \"default\",\n \"system_fingerprint\": \"fp_cbf1785567\"\n}\n",
"headers": []
}
}
Loading
Loading