Databricks Knowledge Assistant

[IM_01]

Hi All,

I've been exploring Knowledge Assistant and trying to understand how it works under the hood. I noticed it's built on the Instructed Retriever architecture, and I recently came across an update called Instructed-Retriever-1, which is supposed to deliver much faster results. Has this already been rolled out to all Databricks accounts, or is it still limited to certain users? Also, curious to understand - what exactly is different about Instructed-Retriever-1 compared to the original Instructed Retriever?

[suryaprayaga]

Yes this is very much implemented, however I haven't tested it much as much I have tested in Grafana's Assistant. The new feature Instructed-Retriever-1 is now conveniently called as Knowledge Assistant. 

How is Instructed-Retriever different from the latest Instructed-Retriever-1 a.k.a Knowledge Assistant?

Instructed-Retriever is more linear in nature - as in, it does not have any parallel searches, query comparisons etc., However the new Knowledge Assistant can run parallel queries and intelligently combined the evidence output to rank the results to give the best evidence. 

So in terms of quality and model accuracy I would say the new Knowledge Assistant is far better than the older one. However it will be proved eventually on several groundtruth facts validated earlier.