Does Sourcehut plan to support multiple formats for READMEs?
It seems that GitHub renders Org files nicely.
From the GitLab documentation [1] I see that they support plain text, Markdown, reStructuredText, AsciiDoc, Textile, rdoc, Orgmode, creole, Mediawiki. It chooses how to render it based on the file's extension.
I'm mainly interested in Org files. I guess this would largely depend on the availability of parsing libraries for those formats in Python or Go, those being the languages of Sourcehut?
Supporting multiple markup languages could be useful for more than READMEs, of course. Deploying wikis with Org files would be a delight.
[1]: https://docs.gitlab.com/ee/user/project/repository/#supported-markup-languages-and-extensions
Cheers,
Alexandru-Sergiu Marton
https://brown.121407.xyz
[8571 7664 DFDC 6B6A 3ED2 DA68 2348 7CC8 2397 2C1F]
<brown121407@member.fsf.org>
On Sun, 2020-03-08 at 17:32 +0000, Alexandru-Sergiu Marton wrote:
> I'm mainly interested in Org files. I guess this would largely depend> on the availability of parsing libraries for those formats in Python> or Go, those being the languages of Sourcehut?
For org specifically, it should be straightforward to incorporate a
local commit hook to export org files to markdown. Unless the project is
somehow associated with Emacs, that may be unduly burdensome upon other
contributors. For small projects or those where Emacs use can be
assumed, this may be a suitable approach.
-gb
On 2020-03-08 15:22, Greg B wrote:
> On Sun, 2020-03-08 at 17:32 +0000, Alexandru-Sergiu Marton wrote:>> I'm mainly interested in Org files. I guess this would largely depend>> on the availability of parsing libraries for those formats in Python>> or Go, those being the languages of Sourcehut?> > For org specifically, it should be straightforward to incorporate a> local commit hook to export org files to markdown. Unless the project > is> somehow associated with Emacs, that may be unduly burdensome upon other> contributors. For small projects or those where Emacs use can be> assumed, this may be a suitable approach.> > -gb
I just wanted to share that I tried this workaround and it did not work
so well. By "tried" I mean that I executed org-md-export-to-markdown
from the buffer with my README.org file, then pushed the resulting
README.md straight up to my repo. The results can be seen here:
https://git.sr.ht/~trs-80/rofi-in-elisp
Now, I have not begun really digging into this yet, as this was just an
initial test (and page still needs work, too). However right off the
bat it seems like none of the in-page links (to headings, footnotes,
etc.) seem to be working.
If you were referring to some other way, or know of any better way, I
would be all ears. I searched EmacsWiki as well as MELPA but could not
seem to find anything else in the vein of a direct Org to Markdown
exporter. I guess maybe Pandoc might be my next step, or...?
On the broader question of whether to support Orgmode markup or not,
from what I have read (please correct me if I am wrong) this does not
seem to me like a technical decision so much as an... I dunno... an "I
(Drew) just don't like Orgmode" one?
Whether that's the case or not, I must admit I'm really scratching my
head at the strategic decision to potentially alienate a whole group of
people (not unlike myself) who are likely in very strong alignment with
everything this project stands for.
After searching around for quite some time for a home for my little
projects (and becoming only disgusted by most of large incumbents), I
have been so excited about sr.ht ever since finding it that I
enthusiastically talk it up to anyone who will stand still long enough!
However the whole attitude[0] toward supporting Org markup just sticks
in my craw, not to mention on the tactical level just being a bit of a
hassle at the moment (assuming I eventually can figure out some
workaround).
Regards,
TRS-80
[0] Perhaps wrongly percieved on my part?
On 11/29/20 1:03 AM, sourcehut.trs-80@isnotmyreal.name wrote:
> On 2020-03-08 15:22, Greg B wrote:>> On Sun, 2020-03-08 at 17:32 +0000, Alexandru-Sergiu Marton wrote:>>> I'm mainly interested in Org files. I guess this would largely depend>>> on the availability of parsing libraries for those formats in Python>>> or Go, those being the languages of Sourcehut?>>>> For org specifically, it should be straightforward to incorporate a>> local commit hook to export org files to markdown. Unless the project is>> somehow associated with Emacs, that may be unduly burdensome upon other>> contributors. For small projects or those where Emacs use can be>> assumed, this may be a suitable approach.>>>> -gb> > I just wanted to share that I tried this workaround and it did not work> so well. By "tried" I mean that I executed org-md-export-to-markdown> from the buffer with my README.org file, then pushed the resulting> README.md straight up to my repo. The results can be seen here:> > https://git.sr.ht/~trs-80/rofi-in-elisp> > Now, I have not begun really digging into this yet, as this was just an> initial test (and page still needs work, too). However right off the> bat it seems like none of the in-page links (to headings, footnotes,> etc.) seem to be working.
Looks like the inserted <a id=""> HTML markup is somehow getting
dropped. I think this may be due to the bleach sanitizer configuration
not whitelisting this for anything other than headers.
That being said, it's not clear to me why your org mode exporter *wants*
to drop people into links like this:
ToC:
```
1. [rofi-in-elisp](#orgdbf2274)
```
Body:
```
<a id="orgdbf2274"></a>
# rofi-in-elisp
```
when markdown already defines this header to work as:
https://git.sr.ht/~trs-80/rofi-in-elisp/tree/master/README.md#rofi-in-elisp> If you were referring to some other way, or know of any better way, I> would be all ears. I searched EmacsWiki as well as MELPA but could not> seem to find anything else in the vein of a direct Org to Markdown> exporter. I guess maybe Pandoc might be my next step, or...?> > On the broader question of whether to support Orgmode markup or not,> from what I have read (please correct me if I am wrong) this does not> seem to me like a technical decision so much as an... I dunno... an "I> (Drew) just don't like Orgmode" one?> > Whether that's the case or not, I must admit I'm really scratching my> head at the strategic decision to potentially alienate a whole group of> people (not unlike myself) who are likely in very strong alignment with> everything this project stands for.> > After searching around for quite some time for a home for my little> projects (and becoming only disgusted by most of large incumbents), I> have been so excited about sr.ht ever since finding it that I> enthusiastically talk it up to anyone who will stand still long enough!> However the whole attitude[0] toward supporting Org markup just sticks> in my craw, not to mention on the tactical level just being a bit of a> hassle at the moment (assuming I eventually can figure out some> workaround).> > Regards,> > TRS-80> > [0] Perhaps wrongly percieved on my part?
There is no attitude towards Org markup.
Drew merely decided that there would be exactly one supported rendering
format, and that is markdown, because he didn't want to get into the
business of supporting the favorite format of everyone and his dog.
If a second supported format were to exist, python's .rst or perl's .pod
would likely be implemented before Org markup, due to being rather more
popular... but does that mean Drew personally hates on .rst or .pod?
Instead, you're supposed to generate your readme contents by
post-processing "whichever special format you use" into HTML, then
setting the repository to use it via the API:
https://man.sr.ht/git.sr.ht/#setting-a-custom-readme
This supports your own finely crafted HTML, Org mode, .rst, .pod,
markdown (if for some reason you feel the need to process it yourself),
or absolutely anything else.
You can e.g. set a builds.sr.ht integration to do this for you on every
commit.
--
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User
> On 2020-11-29 01:56, Eli Schwartz wrote:>> On 11/29/20 1:03 AM, TRS-80 wrote:>> >> I just wanted to share that I tried this workaround and it did not >> work>> so well. By "tried" I mean that I executed org-md-export-to-markdown>> from the buffer with my README.org file, then pushed the resulting>> README.md straight up to my repo. The results can be seen here:>> >> https://git.sr.ht/~trs-80/rofi-in-elisp>> >> Now, I have not begun really digging into this yet, as this was just >> an>> initial test (and page still needs work, too). However right off the>> bat it seems like none of the in-page links (to headings, footnotes,>> etc.) seem to be working.> > Looks like the inserted <a id=""> HTML markup is somehow getting> dropped. I think this may be due to the bleach sanitizer configuration> not whitelisting this for anything other than headers.
FWIW, I noticed this, too, before posting my original email to the list.
However thanks for confirmation. After looking into this a lot more
since then, it does indeed seem that this is the problem.
Is there some (security or other?) reason why the id attribute is
sanitized from anchor links?
> Instead, you're supposed to generate your readme contents by> post-processing "whichever special format you use" into HTML, then> setting the repository to use it via the API:> https://man.sr.ht/git.sr.ht/#setting-a-custom-readme> > This supports your own finely crafted HTML, Org mode, .rst, .pod,> markdown (if for some reason you feel the need to process it> yourself), or absolutely anything else.
I have been looking in to all possibilities (including HTML export,
pandoc, etc.) for a couple days now.
Apparently all of Orgmode exports (Markdown as well as HTML) rely on
that id attribute extensively. In fact, Markdown exporter (ox-md) is
derived from the HTML exporter, and per Markdown spec should allow HTML
to pass through. But then the id attribute gets sanitized out...
I also discovered some other problem[0] trying to go the pandoc route.
Overall, this has turned into quite a lot more effort than what I would
have expected.
Cheers,
TRS-80
[0]
https://groups.google.com/g/pandoc-discuss/c/D-8J4RGiYsk/m/g45AutiNAAAJ
On Wed Dec 2, 2020 at 5:34 PM EST, TRS-80 wrote:
> > On 2020-11-29 01:56, Eli Schwartz wrote:> >> On 11/29/20 1:03 AM, TRS-80 wrote:> >> > >> I just wanted to share that I tried this workaround and it did not > >> work> >> so well. By "tried" I mean that I executed org-md-export-to-markdown> >> from the buffer with my README.org file, then pushed the resulting> >> README.md straight up to my repo. The results can be seen here:> >> > >> https://git.sr.ht/~trs-80/rofi-in-elisp> >> > >> Now, I have not begun really digging into this yet, as this was just > >> an> >> initial test (and page still needs work, too). However right off the> >> bat it seems like none of the in-page links (to headings, footnotes,> >> etc.) seem to be working.> > > > Looks like the inserted <a id=""> HTML markup is somehow getting> > dropped. I think this may be due to the bleach sanitizer configuration> > not whitelisting this for anything other than headers.>> FWIW, I noticed this, too, before posting my original email to the list.> However thanks for confirmation. After looking into this a lot more> since then, it does indeed seem that this is the problem.>> Is there some (security or other?) reason why the id attribute is> sanitized from anchor links?
It's just an oversight. We use a whitelist so we have to add everything
we want. Patches welcome.
On 2020-12-03 11:07, Drew DeVault wrote:
> On Wed Dec 2, 2020 at 5:34 PM EST, TRS-80 wrote:>> > On 2020-11-29 01:56, Eli Schwartz wrote:>> >>> > Looks like the inserted <a id=""> HTML markup is somehow getting>> > dropped. I think this may be due to the bleach sanitizer configuration>> > not whitelisting this for anything other than headers.>> >> Is there some (security or other?) reason why the id attribute is>> sanitized from anchor links?> > It's just an oversight. We use a whitelist so we have to add everything> we want. Patches welcome.
My patch to add the id attribute to anchor links is now live[0], however
as I suspected it did not quite fix the problem.
I made a small change to my README.md and pushed it up to my repo to
make sure it re-generated the HTML, which can be seen:
https://git.sr.ht/~trs-80/rofi-in-elisp
If you look closely at the HTML source, you can see the anchor links are
there, however without any id in them, e.g.:
<p><a rel="nofollow noopener"></a></p>
Which of course render them useless.
I am pretty sure this is because there is no provision to insert the id
when constructing the link in the render_link[1] and/or maybe the
render_plain_link[2] function(s).
Now I have been studying these but my Python is only so-so and I am
afraid solving this particular problem is a bit beyond me at the moment.
Any help would be greatly appreciated. In fact I am willing even to
throw someone a few bucks to implement this.
Cheers,
TRS-80
[0] https://lists.sr.ht/~sircmpwn/sr.ht-dev/patches/15504
[1]
https://git.sr.ht/~sircmpwn/core.sr.ht/tree/master/srht/markdown.py#L67
[2]
https://git.sr.ht/~sircmpwn/core.sr.ht/tree/master/srht/markdown.py#L80
TRS-80 writes:
> I am pretty sure this is because there is no provision to insert the id> when constructing the link in the render_link[1] and/or maybe the> render_plain_link[2] function(s).
The issue is in how the id attribute was added to the sanitizer
whitelist. I've submitted a patch here:
https://lists.sr.ht/~sircmpwn/sr.ht-dev/patches/15867
On 2020-12-08 19:44, Nolan Prescott wrote:
> TRS-80 writes:>> I am pretty sure this is because there is no provision to insert the>> id when constructing the link in the render_link[1] and/or maybe the>> render_plain_link[2] function(s).> > The issue is in how the id attribute was added to the sanitizer> whitelist. I've submitted a patch here:> > https://lists.sr.ht/~sircmpwn/sr.ht-dev/patches/15867
I can confirm this is now working! I pushed a small change in order to
regenerate the HTML to test. Results (working in-page links generated
by org-md-export-to-markdown function) can now be seen at:
https://sr.ht/~trs-80/rofi-in-elisp/
Nolan,
What may have been only a few lines of Python for you, mean a great deal
to me (and I am sure, others who will come later and also prefer .org to
.md).
I thank you so much.
I have several more Elisp/Orgmode based projects that I was holding off
on uploading to sr.ht, because this issue was a near total deal breaker
for me (sorry, but yes I really do hate Markdown that much). However I
love everything else about sr.ht and what Drew is trying to accomplish
here, and so this makes me very, very happy to be able to continue my
participation here.
Please reply to me directly off list with your address (or other
preferred method), and I will be happy to mail you a check, as I am a
man of my word.
---
One final detail. From the beginning, I had been pursuing two different
solutions. One has been resolved here and is now working. The other
was via Pandoc, and has by now been partially solved, with a workaround,
and a full solution seems to be pending. In case anyone coming along
later is interested in going that other route, more info can be found at
the following link:
https://github.com/jgm/pandoc/issues/6916
Cheers,
TRS-80