Obsidian 原生插件管理技术细节研究
1. 缘起
在国内网络环境下，我发现插件、主题在检查更新的时候非常慢，走代理之后就非常顺畅，这让我好奇 Obsidian 的插件管理逻辑是怎么样的。
2. 定性分析
Obsidian Plugin 都是通过 Github 做 Release 的，结合这个条件和上面看到现象可以得到一个结论，这个问题应该是国内用户特有的，本质原因跟 go packages、Rust crates 访问慢的问题是一致的。
由此引发了另外一个问题，如果下载 Obsidian Plugin Release Package 慢可以理解，但是检查更新也缓慢就有点奇怪，是什么原因导致检查更新也变慢了。
3. 定量分析
Obsidian 的技术栈主要是 electron、javascript、typescript 等，涉及到很多的 Web 开发知识，这并不是我擅长的。所以我不打算从代码级别来分析碰到问题，我打算从自己目前掌握的 Web 相关的技术知识尝试分析，试着看能否解决这个问题。
3.1 Obsidian 插件文档
首先从 Obsidian 官方给出的插件开发文档入手，先了解 Obsidian Plugin 开发的基本情况。
3.1.1 Obsidian 插件开发技术要求
- Electron
- Nodejs
- TypeScript
- JavaScript
- Git
- VS Code
3.1.2 Obsidian 社区插件管理流程
- 基础开发环境准备（nodejs、typescript、git、github 账号等）
- 在 github 上 fork Obsidian 提供的 Sample Plugin
- 研发插件，并按照格式维护 manifest.json
- PR 提交插件到 Obsidian 社区进行 review，主要是维护 community-plugin.json 文件
3.2 调试插件管理逻辑
大致思路是通过 DevTools 观查插件更新检查的网络活动，通过网络请求数据结合相关的 javascript 代码分析插件的管理逻辑。
3.2.1 查询插件更新按钮的 HTML 元素
通过 DevTools 的 HTML 元素选择器确认插件更新按钮的 attribute：
确认按钮的 class 属性是 'mod-cta'。
3.2.2 观测插件更新按钮的网络活动
尝试在 DevTools 的 Network Tab 中观察能够捕捉到点击，如下图所示：
很遗憾没有能够捕捉到任何网络活动，不太确定点击检查更新按钮之后的网络活动应该如何监测。
3.2.3 分析插件更新 callback 代码
目前看来想尝试 Debug 分析出插件管理逻辑有点困难，比较幸运的是可以通过 DevTools Application 中查看源码，从过阅读按钮的 callback 源码分析清楚插件管理的逻辑。查看下面的源码阅读 comments:
// 更新插件按钮增加的 callback function,
// 可以看出 checkForUpdates 函数是用来检查
new $A(n).setName(h_.labelCurrentPlugins()).setDesc(h_.labelCurrentlyInstalled({
count: a.length
}) + " " + (l > 0 ? h_.msgUpdatesFound({
count: l
}) : "")).then((function (e) {
l > 0 ? e.addButton((function (e) {
return e.setCta().setButtonText(h_.buttonUpdateAllPlugins()).onClick((function () {
return $k(n, (function () {
return g(t, void 0, void 0, (function () {
var e, t, n, r, o;
return v(this, (function (a) {
switch (a.label) {
case 0:
for (t in e = [],
s)
e.push(t);
n = 0,
a.label = 1;
case 1:
return n < e.length ? (r = e[n],
s.hasOwnProperty(r) ? (o = s[r],
[4, i.installPlugin(o.repo, o.version, o.manifest)]) : [3, 3]) : [3, 4];
case 2:
a.sent(),
a.label = 3;
case 3:
return n++,
[3, 1];
case 4:
return this.render(),
[2]
}
}
))
}
))
}
))
}
))
}
)) : a.length > 0 && e.addButton((function (e) {
return e.setCta().setButtonText(h_.buttonCheckForUpdates()).onClick((function () {
return $k(n, (function () {
return g(t, void 0, void 0, (function () {
return v(this, (function (e) {
switch (e.label) {
case 0:
return [4, i.checkForUpdates()];
case 1:
return e.sent(), this.render(), [2]
}
}
))
}
))
}
))
}
))
}
))
}
))
// Ow 函数的作用是发送 URL Request 请求数据
function Ow(e) {
return Pw(
(function (e) {
return g(this, void 0, Promise, function () {
return v(this, function (t) {
switch (t.label) {
case 0:
return (
String.isString(e) &&
(e = {
url: e,
}),
[4, Lw(e)]
);
case 1:
return [2, t.sent()];
}
});
});
})(e)
);
}
// Lw 函数的作用是拼接 Requet 请求
function Lw(e) {
return g(this, void 0, Promise, function () {
var t, n, i, r, o, a, s, l, c, u, h, p;
return v(this, function (d) {
switch (d.label) {
case 0:
return (
String.isString(e) &&
(e = {url: e,}),
Xb ? ((n = !1), e.body instanceof ArrayBuffer ? ((t = z(e.body)), (n = !0)) : (t = e.body),
[4, Qb.requestUrl({
url: e.url,
method: e.method,
contentType: e.contentType,
headers: e.headers,
body: t,
binary: n,
}),])
: [3, 2]);
case 1:
return ((i = d.sent()), [2, Aw(e, i.status, i.headers, V(i.body))]);
case 2:
return rt("electron") ? [4, st(e)] : [3, 4];
case 3:
return (r = d.sent()), [2, Aw(e, r.status, r.headers, r.body)];
case 4:
return (
(o = e.url), (a = e.method), (s = e.contentType), (l = e.body), (c = null), s &&(c = {"Content-Type": s,}), [4, fetch(o, {method: a, headers: c, body: l,}),]
);
case 5:
return [4, (u = d.sent()).arrayBuffer()];
case 6:
return ((h = d.sent()), (p = {}), u.headers.forEach(function (e, t) {return (p[t] = e);}), [2, Aw(e, u.status, p, h)]);
}
});
});
}
// checkForUpdates 函数安全检查更新的类型来发送检查更新的请求，
// 通过 SD 函数发送请求获取文件 github.com/obsidianmd/obsidian-releases/community-plugins.json
// 通过 rk 函数发送请求获取插件 manifest.json 文件，https://raw.githubusercontent.com/HEAD/{plugin-repo-release}
e.prototype.checkForUpdates = function () {
return g(this, void 0, void 0, (function () {
var e, t, n, i, r, o, a, s, l, c, u, h, p, d;
return v(this, (function (f) {
switch (f.label) {
case 0:
this.updates = {}, e = this.updates, f.label = 1;
case 1:
return f.trys.push([1, 3, , 4]), [4, SD()];
case 2:
return t = f.sent(), [3, 4];
case 3:
return n = f.sent(), console.error(n), new aD(xD.msgFailedLoadPlugins()), [2];
case 4:
i = 0, r = t, f.label = 5;
case 5:
if (!(i < r.length)) return [3, 12];
if (o = r[i], a = o.id, !this.manifests.hasOwnProperty(a)) return [3, 11];
if (s = this.manifests[a], !(l = s.version)) return [3, 11];
c = o.repo, u = rk(c, lD), h = null, f.label = 6;
case 6:
return f.trys.push([6, 8, , 9]), [4, Ow(u).json];
case 7:
return (h = f.sent()) && h.id && h.id === a ? [3, 9] : [3, 11];
case 8:
return f.sent(), [3, 11];
case 9:
return [4, sk(c, h)];
case 10:
(p = f.sent()) && zw(l, p) && (e[a] = {repo: c, version: p, manifest: h}), f.label = 11;
case 11:
return i++, [3, 5];
case 12:
return d = Object.keys(e).length, new aD(0 === d ? xD.msgNoUpdatesFound() : xD.msgUpdatesFound({count: d})),[2]
}
}
))
}
))
}
3.3 插件管理逻辑总结
- obsidian 将读取 community-plugins.json 中的插件列表。
- 插件的 name、author、description 字段用于搜索。
- 当用户打开插件的详细信息页面时，Obsidian 将从对应的 GitHub 存储库中提取 manifest.json 和 README.md。
- 插件 repo 中的 manifest.json 将仅用于找出最新版本，实际的文件是从插件的 GitHub Release 中获取的。
- 如果 manifest.json 需要比正在运行的应用程序更高的 Obsidian 版本，obsidian 的 versions.json 将被查询以找到兼容的最新版本的插件。
- 当用户选择安装您的插件时，Obsidian 将查找与 manifest.json 中的版本标记相同的 GitHub 版本。
- Obsidian 将下载 manifest.json、main.js 和 styles.css（如果可用），并将它们存储在保险库内的适当位置。
- 当点击插件更新检查按钮的时候，Obsidian 会根据本地的插件列表遍历对应插件 repo 中 manifest.json 以确实当前本地的插件是否是最新版本。
4. 社区方案分析
关于 obsidian 插件管理有一个插件解决方案：TfTHacker/obsidian42-brat: BRAT，虽然它的初衷是帮助做插件 beta 版本的测试管理，不过里面也涉及到了插件自动更新管理的事情，这里做一个简单分析总结（主要是阅读源码 BetaPlugins.ts）：
- BRAT 插件更新检查相关的函数调用链：
| -- checkForUpdatesAndInstallUpdates() ---->
| -- updatePlugin() ---->
| -- addPlugin() ---->
| -- |
| -- | -- this.plugin.app.vault.adapter.read() ---->
| -- | -- getRelease() ---->
| -- | -- |
| -- | -- | -- getAllReleaseFiles() ---->
| -- | -- | -- |
| -- | -- | -- | -- grabReleaseFileFromRepository()
2. 检查是否需要更新的关键逻辑是：对比本地的 manifest.json(manifest-beta.json) 文件中的版本与 Github Release 中的 manifest.json 的 version 字段，从而确实是否需要更新（根据上面的函数调用链结合源码阅读非常清楚的看到 BRAT 管理插件更细的逻辑）。
/**
* walks through the list of plugins without frozen version and performs an update
*
* @param {boolean} showInfo should this with a started/completed message - useful when ran from CP
* @return {Promise<void>}
*/
async checkForUpdatesAndInstallUpdates(showInfo = false, onlyCheckDontUpdate = false): Promise<void> {
if(await isConnectedToInternet()===false) {
console.log("BRAT: No internet detected.")
return;
}
let newNotice: Notice;
const msg1 = `Checking for plugin updates STARTED`;
this.plugin.log(msg1, true);
if (showInfo && this.plugin.settings.notificationsEnabled) newNotice = new Notice(`BRAT\n${msg1}`, 30000);
const pluginSubListFrozenVersionNames =
new Set(this.plugin.settings.pluginSubListFrozenVersion.map(f => f.repo));
for (const bp of this.plugin.settings.pluginList) {
if (pluginSubListFrozenVersionNames.has(bp)) {
continue;
}
await this.updatePlugin(bp, onlyCheckDontUpdate);
}
const msg2 = `Checking for plugin updates COMPLETED`;
this.plugin.log(msg2, true);
if (showInfo) {
newNotice.hide();
ToastMessage(this.plugin, msg2, 10);
}
}
/**
* updates a beta plugin
*
* @param {string} repositoryPath repository path on GitHub
* @param {boolean} onlyCheckDontUpdate only looks for update
*
* @return {Promise<void>}
*/
async updatePlugin(repositoryPath: string, onlyCheckDontUpdate = false, reportIfNotUpdted = false): Promise<boolean> {
const result = await this.addPlugin(repositoryPath, true, onlyCheckDontUpdate, reportIfNotUpdted);
if (result === false && onlyCheckDontUpdate === false)
ToastMessage(this.plugin, `${repositoryPath}\nUpdate of plugin failed.`)
return result;
}
/**
* Primary function for adding a new beta plugin to Obsidian.
* Also this function is used for updating existing plugins.
*
* @param {string} repositoryPath path to GitHub repository formated as USERNAME/repository
* @param {boolean} updatePluginFiles true if this is just an update not an install
* @param {boolean} seeIfUpdatedOnly if true, and updatePluginFiles true, will just check for updates, but not do the update. will report to user that there is a new plugin
* @param {boolean} reportIfNotUpdted if true, report if an update has not succed
* @param {string} specifyVersion if not empty, need to install a specified version instead of the value in manifest{-beta}.json
*
* @return {Promise<boolean>} true if succeeds
*/
async addPlugin(repositoryPath: string, updatePluginFiles = false, seeIfUpdatedOnly = false, reportIfNotUpdted = false, specifyVersion = ""): Promise<boolean> {
const noticeTimeout = 10;
let primaryManifest = await this.validateRepository(repositoryPath, true, false); // attempt to get manifest-beta.json
const usingBetaManifest: boolean = primaryManifest ? true : false;
if (usingBetaManifest === false)
primaryManifest = await this.validateRepository(repositoryPath, false, true); // attempt to get manifest.json
if (primaryManifest === null) {
const msg = `${repositoryPath}\nA manifest.json or manifest-beta.json file does not exist in the root directory of the repository. This plugin cannot be installed.`;
this.plugin.log(msg, true);
ToastMessage(this.plugin, `${msg}`, noticeTimeout);
return false;
}
if (!primaryManifest.hasOwnProperty('version')) {
const msg = `${repositoryPath}\nThe manifest${usingBetaManifest ? "-beta" : ""}.json file in the root directory of the repository does not have a version number in the file. This plugin cannot be installed.`;
this.plugin.log(msg, true);
ToastMessage(this.plugin, `${msg}`, noticeTimeout);
return false;
}
// Check manifest minAppVersion and current version of Obisidan, don't load plugin if not compatible
if(primaryManifest.hasOwnProperty('minAppVersion')) {
if( !requireApiVersion(primaryManifest.minAppVersion) ) {
const msg = `Plugin: ${repositoryPath}\n\n`+
`The manifest${usingBetaManifest ? "-beta" : ""}.json for this plugin indicates that the Obsidian ` +
`version of the app needs to be ${primaryManifest.minAppVersion}, ` +
`but this installation of Obsidian is ${apiVersion}. \n\nYou will need to update your ` +
`Obsidian to use this plugin or contact the plugin developer for more information.`;
this.plugin.log(msg, true);
ToastMessage(this.plugin, `${msg}`, 30);
return false;
}
}
const getRelease = async () => {
const rFiles = await this.getAllReleaseFiles(repositoryPath, primaryManifest as PluginManifest, usingBetaManifest, specifyVersion);
if (usingBetaManifest || rFiles.manifest === "") //if beta, use that manifest, or if there is no manifest in release, use the primaryManifest
rFiles.manifest = JSON.stringify(primaryManifest);
if (rFiles.mainJs === null) {
const msg = `${repositoryPath}\nThe release is not complete and cannot be download. main.js is missing from the Release`;
this.plugin.log(msg, true);
ToastMessage(this.plugin, `${msg}`, noticeTimeout);
return null;
}
return rFiles;
}
if (updatePluginFiles === false) {
const releaseFiles = await getRelease();
if (releaseFiles === null) return false;
await this.writeReleaseFilesToPluginFolder(primaryManifest.id, releaseFiles);
await addBetaPluginToList(this.plugin, repositoryPath, specifyVersion);
//@ts-ignore
await this.plugin.app.plugins.loadManifests();
const versionText = specifyVersion === "" ? "" : ` (version: ${specifyVersion})`;
const msg = `${repositoryPath}${versionText}\nThe plugin has been registered with BRAT. You may still need to enable it the Community Plugin List.`;
this.plugin.log(msg, true);
ToastMessage(this.plugin, msg, noticeTimeout);
} else {
// test if the plugin needs to be updated
// if a specified version is provided, then we shall skip the update
const pluginTargetFolderPath = this.plugin.app.vault.configDir + "/plugins/" + primaryManifest.id + "/";
let localManifestContents = "";
try {
localManifestContents = await this.plugin.app.vault.adapter.read(pluginTargetFolderPath + "manifest.json")
} catch (e) {
if (e.errno === -4058 || e.errno === -2) { // file does not exist, try installing the plugin
await this.addPlugin(repositoryPath, false, usingBetaManifest, false, specifyVersion);
return true; // even though failed, return true since install will be attempted
}
else
console.log("BRAT - Local Manifest Load", primaryManifest.id, JSON.stringify(e, null, 2));
}
if (
specifyVersion !== ""
|| this.plugin.settings.pluginSubListFrozenVersion.map(x=>x.repo).includes(repositoryPath)
) {
// skip the frozen version plugin
ToastMessage(this.plugin, `The version of ${repositoryPath} is frozen, not updating.`, 3);
return false;
}
const localManifestJSON = await JSON.parse(localManifestContents);
if (localManifestJSON.version !== primaryManifest.version) { //manifest files are not the same, do an update
const releaseFiles = await getRelease();
if (releaseFiles === null) return false;
if (seeIfUpdatedOnly) { // dont update, just report it
const msg = `There is an update available for ${primaryManifest.id} from version ${localManifestJSON.version} to ${primaryManifest.version}. `;
this.plugin.log(msg + `[Release Info](https://github.com/${repositoryPath}/releases/tag/${primaryManifest.version})`, false);
ToastMessage(this.plugin, msg, 30, async () => { window.open(`https://github.com/${repositoryPath}/releases/tag/${primaryManifest.version}`)});
} else {
await this.writeReleaseFilesToPluginFolder(primaryManifest.id, releaseFiles);
//@ts-ignore
await this.plugin.app.plugins.loadManifests();
//@ts-ignore
if (this.plugin.app.plugins.plugins[primaryManifest.id]?.manifest) await this.reloadPlugin(primaryManifest.id); //reload if enabled
const msg = `${primaryManifest.id}\nPlugin has been updated from version ${localManifestJSON.version} to ${primaryManifest.version}. `;
this.plugin.log(msg + `[Release Info](https://github.com/${repositoryPath}/releases/tag/${primaryManifest.version})`, false);
ToastMessage(this.plugin, msg, 30, async () => { window.open(`https://github.com/${repositoryPath}/releases/tag/${primaryManifest.version}`) } );
}
} else
if (reportIfNotUpdted) ToastMessage(this.plugin, `No update available for ${repositoryPath}`, 3);
}
return true;
}
/**
* Gets all the release files based on the version number in the manifest
*
* @param {string} repositoryPath path to the GitHub repository
* @param {PluginManifest<ReleaseFiles>} manifest manifest file
* @param {boolean} getManifest grab the remote manifest file
* @param {string} specifyVersion grab the specified version if set
*
* @return {Promise<ReleaseFiles>} all relase files as strings based on the ReleaseFiles interaface
*/
sync getAllReleaseFiles(repositoryPath: string, manifest: PluginManifest, getManifest: boolean, specifyVersion = ""): Promise<ReleaseFiles> {
const version = specifyVersion === "" ? manifest.version : specifyVersion;
// if we have version specified, we always want to get the remote manifest file.
const reallyGetManifestOrNot = getManifest || (specifyVersion !== "");
return {
mainJs: await grabReleaseFileFromRepository(repositoryPath, version, "main.js", this.plugin.settings.debuggingMode),
manifest: reallyGetManifestOrNot ? await grabReleaseFileFromRepository(repositoryPath, version, "manifest.json", this.plugin.settings.debuggingMode) : "",
styles: await grabReleaseFileFromRepository(repositoryPath, version, "styles.css", this.plugin.settings.debuggingMode)
}
}
/**
* pulls from github a release file by its version number
*
* @param {string} repository path to GitHub repository in format USERNAME/repository
* @param {string} version version of release to retrive
* @param {string<string>} fileName name of file to retrieve from release
*
* @return {Promise<string>} contents of file as string from the repository's release
*/
export const grabReleaseFileFromRepository = async (repository: string, version: string, fileName: string, debugLogging = true): Promise<string|null> => {
const URL = `https://github.com/${repository}/releases/download/${version}/${fileName}`;
try {
const download = await request({ url: URL });
return ((download === "Not Found" || download === `{"error":"Not Found"}`) ? null : download);
} catch (error) {
if(debugLogging) console.log("error in grabReleaseFileFromRepository", URL, error)
return null;
}
}
5. 问题总结
obsidian 检查是否有插件需要更新之所以缓慢，是因为在检查更新的过程中涉及到访问如下域名：
- raw.githubcontent.com
- github.com
这些域名访问会有网络速度很慢的问题，所以解决前文提到的检查插件更新慢的原因需要从这个根因入手。
6. what's next
- 解决 obsidian 自带的插件管理逻辑检查更新和下载插件慢的问题；
- 解决强迫症患者问题，老是想点一下看看有没有插件需要更新，例如自动更新；
- 增加回滚插件版本的能力支持；