Forum / Wiki for Life Itself: combine github discussions with a markdown-based git approach

[rufuspollock]

v2 - 2024-02-26

This proposal is about creating a space for collaborative knowledge building long-term. For now, we are terming this a collective digital garden 🌱.

The proposed approach mixes a knowledgebase/wiki with a ideas playground/forum. The former is more formal and the latter more relaxed and playful.

The proposal builds upon the discussion forum we already have and digital garden experience with our Flowershow tool.

Context

• We have an increasing amount of useful material published on github discussions
• Have some wiki like material in lifeitself.org/notes (that is largely invisible)

Problem

• We're not clear what we need
• Github discussions don't work as a collaborative knowledge base e.g. limited ability to edit each other, poor linking. This is starting to be noticeable as related points are spread across multiple discussions
• KB section of main website (https://lifeitself.org/notes) don't work very well in sense that not very discoverable and not very updated. Also being part of the main site there is a sense they should be higher quality or something like that.

Questions: what do we need and how we pilot it?

What we want

Note

I'm going to speak in the first person for now though obviously the intention is to check these points with others.

The two key features:

• The informality that gives me and others the permission post things that are half-formed or to ask questions.
• And the structure and process to evolve a proper collaborative knowledgebase with more refined entries and rich linking

Other desirables:

• Be able to feature articles and to see what's new (in both areas of the half-formed and the more refined)
• Open but moderated contribution
• Ability to have discussions / questions as well as knowledge curation
• Allow for discussions and comments (for engagement/corrections)

As I reflect on the inspirations below e.g. Wikipedia, LessWrong I see that the primary function for me is a collaborative knowledgebase:

• Easy to create entries
• Easy to link them
• Multiple people can edit an article - but with moderation
• Can have comments / discussion - like a talk page or in a dedicated section of the site

Secondary purpose is a place to ask questions, chat, try out ideas.

Having both is key i think.

[!aside]

Often think a forum is what you have when you can't have annotated comments on the post itself using quotes instead of annotations. Though only partial because difficult to make annotation comments that cover several points across the piece.

You also want to think about new-joiners. They want a place where they can see -- and perhaps contribute -- in a low-stakes way. For example, see the best content. Leave an upvote or comment etc.

I like the idea of mixing a knowledgebase with a playground. The latter is more formal and the latter relaxed and, well, playful.

The evolution of a piece of work

• Start in playground
• Refine into an entry in the knowledgebase
• Eventually a snapshot may become released on the blog
  • NB: one of the advantages of blogs is that they are timestamped and when it's out it's out. You aren't trying to come back to it.

Technical desiderata

• Can edit locally preferably with something like Obsidian
• Moderation for contribution e.g. pull requests would work
• Write in markdown (allows for long-term maintenance, easy conversion etc)

Open questions

• How do you handle comments on different versions. Take the example of this proposal which had a quite different v1 on discussion forum: https://github.com/orgs/life-itself/discussions/992. Now have revised it substantially. Hacky solutions like comments having a statement like: comment on vXXX. Other approach is pull requests with comments there (but then things aren't published).

Concrete steps

Evolve from where we are by combining github discussions with a markdown-based obsidian/git approach and published with Flowershow.

Concretely

• Create a repo like github.com/life-itself/forum (or even reuse community repo moving website out of community repo). This git repo houses markdown files which are the wiki pages / forum posts.
  • every page is automatically discussable with discussion being on github discussions via giscus integration
• people can still post directly on github discussions though may want some way to automate converting discussion body into markdown but that's open to debate

Rabbit holes / Risks

TODO

No goes

TODO

Appendix: Other options considered

• Use discourse (we had a discourse forum from 2016-2019) ❌ it's a forum not a kb/wiki primarily. plus cost, format lock-in
  • 👍 easy to use
  • 👎 recurring cost of ~1k a year.
  • 👎 a forum not a wiki
  • 👎 locked into their system and format (even if open source). If you ever want to leave you are kind of stuck.
• just use github discussions: Go all in on github discussions: just use that and move everything to that. Maybe have a landing page somewhere that explains how we use it. ❌ limited wiki functionality.
  • Works decently as a forum
  • However, as wiki have issue of not having very good linking
  • Can't use obsidian etc directly
  • You are "on" github rather than your own space both in terms of theme, and in terms of things like signing up etc

Appendix: Inspirations

Less Wrong

• Posts are like decent essays
• Substantial commenting feature
• Curated core content
• Forum creates community (feels a bit different from a wiki)

What I miss in lesswrong

• Everything feels pretty polished - it is a high quality forum. I want somewhere i can post things that half-formed etc. See Fowler on his bliki

  I intend to use this to post ideas that are forming, but either too immature or too short for a proper article. Also as I see questions posted on mailing lists or newsgroups I'll try to provide a lasting answer here.

• Also miss collaborative wiki aspect: everything is a new post so little sense of cumulation ...

Wikipedia

• You have the actual articles and the talk pages
• You can have stubs and an evolution in the status and quality of the articles

What i miss

• Discussions or Q&A that aren't related to a topic. though perhaps one gets that in the talk.
• Talk is a very second class citizen. More for debating.

Martin Fowler's Bliki

https://martinfowler.com/bliki/WhatIsaBliki.html

Beyond the name, however, there's the very ephemeral nature of blog postings. Short bursts of writing that might be interesting when they are read - but quickly age. I find writing too hard to want to spend it on things that disappear.

I have similar mixed feelings about wikis. I like the way they allow you to quickly put stuff together. But they can easily lead to long rambling sites. And I do like the fact that blogs make it easy to see what's really changed recently - thanks to the hooks into RSS and aggregators.

So I decided I wanted something that was a cross between a wiki and a blog - which Ward Cunningham immediately dubbed a bliki. Like a blog, it allows me to post short thoughts when I have them. Like a wiki it will build up a body of cross-linked pieces that I hope will still be interesting in a year's time.

I intend to use this to post ideas that are forming, but either too immature or too short for a proper article. Also as I see questions posted on mailing lists or newsgroups I'll try to provide a lasting answer here.

Appendix: wikis, forums, and blogs

• Wiki: A collaborative website where anyone can edit and add content to existing pages.
  • Building and maintaining shared knowledge bases (e.g., Wikipedia)
  • Collaborative documentation projects
  • Internal company wikis for sharing policies and procedures
• Forum: An online discussion board where people can post messages, reply to threads, and engage in discussions about specific topics.
  • Support communities for products or services
  • Online communities with shared interests
  • Q&A platforms for specific topics
• Blog: A regularly updated webpage where individuals or organizations publish content in reverse chronological order (latest post first).
  • Sharing personal thoughts and experiences
  • Providing updates on specific topics or projects
  • Publishing news and commentary

Which one to use when?

Consider factors like:

• Content audience: Is it primarily one-way communication (blog) or two-way discussion (forum/wiki)?
• Content ownership: Do you want open collaboration (wiki) or controlled contributions (blog)?
• Content structure: Does the information require a specific hierarchy (wiki) or chronological order (blog)?

These are not mutually exclusive. We can combine elements of each platform to create a hybrid solution that best suits our needs.

Appendix: SCQA Brainstorm

Situation

• We have an ad-hoc informal forum in the form of github discussions running since ~ late 2022
• Have a notes/wiki/forum section at lifeitself.org/notes (only Rufus has ever really posted there)
• Have old analysis (from 2021/2022) of why we wanted a forum and wiki and why try an integrated site https://lifeitself.org/notes/projects/digital-garden-2022
• Back in the day (2016 or so!) we have a private discourse based forum which we ultimately shuttered around 2019

Complication

• Github discussions don't work as a collaborative knowledge base e.g. limited ability to edit each other, poor linking.
• KB section of main website (https://lifeitself.org/notes) don't work very well in sense that not very discoverable and not very updated
  • Also being part of the main site there is a sense they should be higher quality or something like that.
• Overall building the wiki/forum directly into the main site does not work so well ...
  • Technical reasons:
    • Bloats main site and slows build (?)
    • Build delay: when posting on forum i often want to post and immediately share. Atm i have to wait 2-3m for site to build (if it builds)
  • Social reasons: stuff on main site wants to be more polished.
    • Even I feel a bit reluctant to post half-formed things
    • And if i feel that about .
    • 🔮 Want somewhere with a bit of a rough feel where people feel comfortable throwing stuff up
• The supposed benefits of keeping them in one repo was that we could make references but i think that is quite minor actually.

Questions

What is the need?

• Something wiki like where we can post post stubs, half-finished things etc for reference and discussion
• A place for discussion

[asimong]

I've just tried out the first two tutorials from https://makeitmarkdown.flowershow.app/ and, though I see it as a great idea in principle, unfortunately I think this is only a practical solution for people who already are familiar with GitHub, and hence probably developers. The extra burden compared with most wikis seems to me much too much.

I'm still really keen on helping to select or develop something that serves as a knowledge base, whether or not it is strictly a wiki. Let's take the conversation forward from here...

[rufuspollock]

I've just tried out the first two tutorials from https://makeitmarkdown.flowershow.app/ and, though I see it as a great idea in principle, unfortunately I think this is only a practical solution for people who already are familiar with GitHub, and hence probably developers. The extra burden compared with most wikis seems to me much too much.

IME now non-developers can edit on github even if they aren't familiar with it.

[asimong]

it's not GitHub by itself. It's the whole rest of the paraphernalia which will be pretty daunting

[rufuspollock]

@asimong a normal contributor won't need to touch flowershow or any of that stuff. But they do either need to edit on github or in obsidian and then push. the real obstacle in my experience (which is not quite substantial) is that it is git and pull requests that is the pain point ...

[asimong]

good point, and the language ("commit") is unfamiliar and off-putting. But if you want comment control, everyone will need their own repo, no?

[rufuspollock]

@asimong you can push direct to a branch (but you need to be a contributor).

Plus you can do "fork and pull" directly via the github interface.

And even if you have your own repo you don't need flowershow etc. You just need that for the pull requests.

[asimong]

Here's my work-in-progress review of the tutorial so far.
https://li-tutorial-asg.flowershow.app/

Only very minor points about the tutorial itself — the major critique is not about the tutorial (which does nearly the best it could) but of the system it is introducing.

[rufuspollock]

@asimong this is fantastic - thank-you! I'm reading through.

/cc @elisapaka

[elisapaka]

Thank you so much Simon for taking the time to review! Much appreciated. I'll have a read through at the first opportunity.

[martinprogrammer]

Here's my work-in-progress review of the tutorial so far. https://li-tutorial-asg.flowershow.app/

Only very minor points about the tutorial itself — the major critique is not about the tutorial (which does nearly the best it could) but of the system it is introducing.

@asimong For what it's worth - I've setup a Flowershow Obsidian workflow, but I'm not using it inside Github discussions - I'm simply publishing it on Vercel and it works fine. I'm not using Flowershow integration inside Obsidian, but I'm compiling the markdown pages in the code and then pushing to Github using CLI. Github has continuous deployment to Vercel. https://slantpositive.vercel.app/

Is your issue that the markdown files don't render correctly inside Github?
Making a video tutorial would foolproof the process and quickly highlight any errors or missing steps.

[asimong]

thanks Martin @martinprogrammer — it's not just one thing that is the issue by itself, it's the cumulative cognitive demand on the average user. happy to talk through sometime

[asimong]

Simon's comments on v0.2-alpha

I guess, just some ideas that arise in response … (note, this was on a previous version)

• I can see where this might go with a semantic wiki, but the software isn't fully up to it yet.
  • There would be different kinds of page, flagged/tagged with some level of play/serious
  • There would be different kinds of comments
• What about when ideas are of mixed quality?
  • That's the nice thing about wikis: each piece is small, so pieces of different quality can be separate on the wiki, but naturally cross-linked
  • So you can have some pieces definitive, some tentative, some playful
• A long-form article, for publishing as a blog post, white paper or whatever, if it's within the same system, can easily refer to / link to the ideas contained.
  • The (wiki) small parts can be remixed into different dated articles.
  • There's no need to keep the articles long term, as all their constituent parts are kept in wiki form, as small linked pieces.
  • The dated article represents a kind of snapshot of the author's understanding and mindset at that time. Good to keep that for future reference.
  • "Release on the blog" could be a simple change of attribute / status. Maybe thinking of "moving" it may not be helpful. This sort-of already happens in software like Wordpress. 💬2024-02-12 👍 Rufus
    • I rate Wordpress as too complex and involved, but it surely has several things to learn from.
• Think of sophisiticated searchability. For that, I would say, we need a Linked Data analogue (or even graph database) with its own query language (and they exist of course)
• What about lower-level information / data even? Surely it would be good to have a (graph? or just Turtle?) database equivalent structure in some kind of linked data mode, and it would be great if this was and integral part of the overall structure:
  • people
  • organisations
  • things
  • events
  • courses
  • jobs, opportunities
  • concepts
  • relationships themselves
  • etc. etc.
• Conversational structure, forum like, is indeed another consideration
  • Not sure we can solve anything without semantic links.
  • Comments need to be independent, but at the same time able to be pointed to parts of a referred text.

I haven't tried yet to compare with my requirements for wikis, but maybe sometime.

[rufuspollock]

This is great Simon. I'd separate "how would we implement" from "what do we want". Right now, i want to try and clarify the "what do we want". Then we can look at "how do we implement".

[laurenwigmore]

• Playground/knowledgebase mix is nice and takes off a sense of pressure
• How many people will actually use this in our community?
• What is the actual need of it? This question came up at different points in the thread but also for myself, for what purpose are you wanting to do this and is it needed now?
• In terms of strategy conversations earlier in the year where does it fit? i.e. is this is a priority now and what is its value/cost?
• Does the wiki idea need to be considered now? I get a sense of lots of items want to be done which is creating a stranglehold.
  • It seems to me the focus should be on the 2R pilot course and getting up to date with releasing of our video and podcast materials and defining our workflows. This will allow our guerilla content to tick over and larger projects can truly be shaped and shipped.

[asimong]

No arguments against right timing! On the need though, I would be more cautious. What do people "need" information for? If we stick too closely to a instrumentalist position, we risk ending up with a mechanistic information system. Do people 'need' Wikipedia?

On the other hand, I've been saying that a knowledge commons only comes to life, in my view, when it serves a community of practice that is actually using the knowledge. But here I'd also want to focus on the wider perspective: when we establish a knowledge commons, we can build it in a way that facilitates learning, and a learning community, or indeed a community of interest, both of which are potential recruiting grounds for the community of practice.

And I'm sure this feeds in somehow to DDS, let's think further about that.

[asimong]

Coming back to the question of a wiki.

Recent exchanges have made it clear to me that it would be useful to have a knowledge commons initially containing relevant published papers, articles, sites, pages, etc. Here are some relevant notes

But I do think it would benefit hugely from an operational, easy-to-use, easy to manage/maintain wiki. I did a quick skim through what has been talked about in the last few years, and leaving aside MediaWiki as it is really complex, I am currently left with three:

• DokuWiki doesn't use Markdown natively, but there is a plugin that allows it. I don't know how compatible the Markdown is.
• wiki.js
• Bookstack (new to me, saw it referred to in a few comments online)
• There is perhaps still an alternative in using a Markdown-based stack. See notes of Rufus and Simon, 2024-04-05 (as linked above, also with followup from Simon)

[asimong]

Another thing that I reckon would help a lot here is to have some real case studies of actual documents, how they developed, and any lessons from that.

[AndreasS2501]

Very interesting!!

[asimong]

please enlarge on what you find interesting, @AndreasS2501 , and any suggestions for moving things on...

[AndreasS2501]

So within weco there are different groups and initiatives. Still it is not that big. One of the initiatives is called the world wise web:
https://weco.io/s/www/posts

I think one of the main drivers for rufus idea was to create a different, kind of information organisation between what people internally of your organisation contribute to content and how that can be consumed or modified from other places.

Having looked at www https://weco.io/p/26536 and https://nanopub.net/ https://archive.org/details/kuhn-metagov-20240320 it feels to me as there is a kind of overlap in what you are trying to do and what www tries to do.

[asimong]

from this I found https://knowledgepixels.com/slides/metagov-nanopubs/ which works better for me than a video...

[asimong]

Referring to the nice contribution of @AndreasS2501 above … nanopubs strikes me as an interesting elaboration on the basic RDF / Semantic web / linked data / knowledge graph concepts. And it might form part of the infrastructure, or at least an approach to some aspects of it. What it lacks, to me, is

1. intuitive usability: rather like RDF itself, ordinary people don't think at that level of granularity and abstraction
2. learnability: there is no mention of structuring or metadata for self- or peer-learning approaches
3. collective responsibility and agency: it seems to remain within an individualist paradigm.

This is very normal, as I have seen very little tech that really reaches into the collective space.
Needless to say, I am of course planning to address all three of the points I raised here.

[AndreasS2501]

I think the importance of "freely" available knowledge bases can not be understated. When Google -search created a “simple” interface it included many presumptions. The question now is can nanopub go beyond the academic use case? And be interesting for “common” people? Maybe to connect different Zettelkasten- from obsidian or logseq, such that the users of these could form networks based on graph relating of hier zks through nanopub?
There are a few Problems open :
1.how can P2P hosting work "good enough"? There is a humongous technical infrastructure for the current corporate client/server based internet. Some part of that is obsolete, but surely not all
2. the search standard, its protocol needs to be stable enough - meta stable. Such that "old peers" can still work for a few years, it has to be mature enough such that the most common use cases are supported reliably people will only use something if it works and acutally has some benefit for them.

Have all of these - in place - the benefit could allow for people find a kind of resonance pattern ( related https://www.amazon.de/Resonanz-Soziologie-Weltbeziehung-taschenbuch-wissenschaft/dp/3518298720/ref=sr_1_1?crid=G121P3INMRN6&dib=eyJ2IjoiMSJ9.vXlFZbC-O4eSTyOT_gYWWGUnBTT7kcZAnOlBeSxodrz5l88nojl8w2uQc_KBO07UhBMuzhGJOXbiM9vU5IvZb7eXjrrd7qLWW-gZOZtazk0Uh3SruNFCCI4R3zULXssTD-NVPRaBnmMtIrOJlurU3ihHOi7PInVj_WPCBNqngYt9SfvUIAZ3ECeyCGR7PTwaQfNSB09DwxUmg4VEaaf930_XFjqneZQTGJ5Wz3AVmPk.dbY9Te3-eBq0TNTVSlyPDPCnHnr4FGgLH5_vZP6T_FI&dib_tag=se&keywords=hartmut+rosa+resonanz&qid=1713791897&sprefix=hartmut+rosa+%2Caps%2C90&sr=8-1 )
One of the strong presumptions of search engines as they exist today is that they are deeply embedded in consumer culture and advertising. It is hard to imagine how much these shaped the relations of people using the internet browser just "looking" for information.

In reality the "search" engine is a kind of search for affordances or opportunities of relationships. Yes money makes "transactions" simpler. But by now it should be clear that we as human beings were severely cut in our ability to communicate with these kinds of things.

[rufuspollock]

FIXED. (for now).

This has been an amazing and very fruitful thread - thank you especially @asimong and @AndreasS2501 for your reflections and additions.

I'm marking this as closed as we now have a working system that we can try out at https://notes.lifeitself.org/ that is powered by Flowershow/DataHub Cloud and runs off the content in the git repo here https://github.com/life-itself/community

See #1089 for the work on this ...

⏭️ I haven't yet enabled discussions but that would be a next step!

[AndreasS2501]

Hi @rufuspollock generally the notes look good. But for example these drawing don't seem to work? https://notes.lifeitself.org/excalidraw/comms-analysis-2022-02-24.excalidraw

[rufuspollock]

Yes, the excalidraw images won't render on their own - but the embeds as svg will work etc.

More info at https://flowershow.app/

[rufuspollock]

UPDATE (2025-01-12): see #1164 for likely update in terms of forum/playground part.

In terms of knowledgebase we now have https://notes.lifeitself.org/ and https://wiki.secondrenaissance.net/