Skip to content

feat(smartpicker): wire NC Assistant Smart Picker bridge#17

Draft
chrip wants to merge 1 commit intomainfrom
feat/connect-nc-ai-backend
Draft

feat(smartpicker): wire NC Assistant Smart Picker bridge#17
chrip wants to merge 1 commit intomainfrom
feat/connect-nc-ai-backend

Conversation

@chrip
Copy link
Copy Markdown

@chrip chrip commented May 6, 2026

Adds the editor<->NC bridge needed for the toolbar's Smart Picker / Nextcloud Assistant button (registered on the OO web-apps side):

editor.js (runs inside the editor iframe):

  • config.events.onRequestSmartPicker forwards the editor's postMessage payload to the host with the user's current selection
  • onAppReady detects window.OCA.Assistant.openAssistantForm and calls docEditor.setSmartPickerEnabled so the toolbar entry grays out when the Assistant app is not loaded
  • editorInsertLink / editorInsertPlainText expose Promise-wrapped hooks the host calls to insert the AI result back at the cursor

listener.js (runs in the NC parent window):

  • onSmartPickerRequest opens window.OCA.Assistant.openAssistantForm with taskType core:text2text and the selected text seeded into prompt/input/text keys (covers chat/text2text/translate/summarize /rewrite); registers an 'Insert into editor' action button that routes the finished task's output back through the editor
Screenshot from 2026-05-05 16-09-55 In the screenshot the upper right Assistant icon (circled in red) opens the default Assistant. It has no connection to the content of the Euro-Office canvas. The new entry "Nextcloud Assistant" under "Insert" main menu (circled in red) of Euro-Office opens the Assistant context aware. If text was marked, the text is copied into the input field of the Assistant.

Things to discuss

  • Usability issue, two Assistant icons doing almost same thing
  • Where to position the "Nextcloud Assistant" menu entry in Euro-Office GUI
  • How deep should the integration be? At the moment, markup is stripped. If a user marks for example a headline or a bold text, opens the Assistant, clicks "translate", the translated text is shown as normal unformatted text. No markup preserved.

To make that feature work, one needs the web-apps counter part:
Euro-Office/web-apps#58

@jospoortvliet
Copy link
Copy Markdown
Member

My cents:

Usability issue, two Assistant icons doing almost same thing

Yeah, maybe don't. Rather inline (select text -> translate) like in Nextcloud Text or Talk. The Assistant menu on the top bar should, ideally, work. Not sure how feasible that is of course.

Where to position the "Nextcloud Assistant" menu entry in Euro-Office GUI

See above, then... But if not feasible in time, what you did looks decent to me. Jan might have different ideas of course ;-)

How deep should the integration be? At the moment, markup is stripped. If a user marks for example a headline or a bold text, opens the Assistant, clicks "translate", the translated text is shown as normal unformatted text. No markup preserved.

Well in an ideal world you can tell the assistant "can you create a pivot table of this data that shows the growth of shoe production over time split by factory" -> we are obviously quite far from that. The closer we get there, the better - but there's so much you can do until June 9 ;-)

Being able to deal with markup would be nice, of course. What Collabora has - if there's an error in a spreadsheet, it can advise on it. Would be nice, too, such errors drive me nuts. And then being able to ask "I need to do X with column AB and AC, how do I do it" and have it give me a formula for that... That is gold already.

In slides, you probably have stuff like - re-arrange, or "make it look nice" or "make it look more professional" and then it makes changes to the design... Or "create a summary slide".
I must say the AI integrations Apple did in Keynote have all been utterly useless to me, so I'm not even sure where to look for inspiration.

But I think the question is rather - what is doable. Pretty sure my ideas above won't fit in the time.

@chrip chrip force-pushed the feat/connect-nc-ai-backend branch from a440598 to f5febaa Compare May 8, 2026 10:52
Comment thread src/editor.js
// The web-apps Gateway delivers the user's current selection via
// event.data.selectedText so the NC Assistant can seed its input
// with the text the user wants to operate on.
const selectedText = event && event.data && event.data.selectedText ? event.data.selectedText : ''
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we do:

const selectedText = event?.data?.selectedText ?? ''

I've not tested that, but I think that is the newer approach

Comment thread src/editor.js Outdated
// Tell the editor whether the NC Assistant app is loaded so it
// can gray out the toolbar entry when the user can't actually
// reach an AI provider.
const assistantAvailable = !!(window.OCA && window.OCA.Assistant && typeof window.OCA.Assistant.openAssistantForm === 'function')
Copy link
Copy Markdown
Member

@moodyjmz moodyjmz May 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

const assistantAvailable = typeof window.OCA?.Assistant?.openAssistantForm === 'function'

typeof check against 'function' returns boolean, so no !! needed

Comment thread src/listener.js
}
this.showSmartPicker = true

const openAssistantForm = window.OCA && window.OCA.Assistant && window.OCA.Assistant.openAssistantForm
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

const openAssistantForm = window.OCA?.Assistant?.openAssistantForm

we can collapse it using the optional chaining operator

Comment thread src/listener.js
OCA.Eurooffice.onRequestInsertImage(event.data.param)
break
case 'editorRequestSmartPicker':
OCA.Eurooffice.onSmartPickerRequest(event.data.param && event.data.param.selectedText)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OCA.Eurooffice.onSmartPickerRequest(event.data.param?.selectedText)

Same deal

@moodyjmz
Copy link
Copy Markdown
Member

moodyjmz commented May 8, 2026

@chrip I added a few minor comments

Adds the editor<->NC bridge needed for the toolbar's Smart Picker /
Nextcloud Assistant button (registered on the OO web-apps side):

editor.js (runs inside the editor iframe):
- config.events.onRequestSmartPicker forwards the editor's
  postMessage payload to the host with the user's current selection
- onAppReady detects window.OCA.Assistant.openAssistantForm and
  calls docEditor.setSmartPickerEnabled so the toolbar entry grays
  out when the Assistant app is not loaded
- editorInsertLink / editorInsertPlainText expose Promise-wrapped
  hooks the host calls to insert the AI result back at the cursor

listener.js (runs in the NC parent window):
- onSmartPickerRequest opens window.OCA.Assistant.openAssistantForm
  with taskType core:text2text and the selected text seeded into
  prompt/input/text keys (covers chat/text2text/translate/summarize
  /rewrite); registers an 'Insert into editor' action button that
  routes the finished task's output back through the editor
- editorRequestSmartPicker case in the postMessage switch
- Falls back to the plain getLinkWithPicker bridge if the Assistant
  app is not installed

Signed-off-by: Christoph Schaefer <christoph.schaefer@nextcloud.com>
@chrip chrip force-pushed the feat/connect-nc-ai-backend branch from f5febaa to a3e0a9c Compare May 8, 2026 21:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants