I host my own locally and set it to restart the docker every hour like they mentioned in their guide.
That’s the part why I stay away from Invidious. Why would you need to restart every hour? How inconvenient :/.
Piped has other issues right now but fixes are underway :)
Yeaaah ! It has some degree of customization via docker environment. Dunno how far you can go though, I only have a sleep interval option.
But the barebone default configuration works out of the box :D!
It’s simple, no UI bloat, can export audio, video and I don’t have to mess with yt-dlp which is huge and the documentation is like reading a whole book xD.
Though barebone yt-dlp allows way more :)
Hey take your time :) Don’t worry even if you forget, you did more than enough to help some random on the web ! 2 other users came up with a plain/bare bone regex solution if you want to have a look and maybe there’s something you can learn out of it? (I doubt it xD).
Plain sed regex (https://lemmy.ml/post/25346014/16453351)
sed -E ':l;s/(\[[^]]*\]\()([^)#]*#[^)]*\))/\1\n\2/;Te;H;g;s/\n//;s/\n.*//;x;s/.*\n//;/^https?:/!{:h;s/^([^#]*#[^)]*)(%20|\.)([^)]*\))/\1-\3/;th;s/(#[^)]*\))/\L\1/;};tl;:e;H;z;x;s/\n//;'
Plain Pearl regex (https://lemmy.ml/post/25346014/16453161)
perl -pe 's/\[[^]]+\]\((?!https?)[^#]*#\K[^)]+(?=\))/lc $&=~s:%20|\d\K\.(?=\d):-:gr/ge'
Nonetheless, I really prefere your solution because as someone else said I will have an easier time to change a script I “understand”. Soo thanks again !
Thank you ! It does actually ticks every use case (for my files) looks pretty rad !
This might work, but I think it is best to not tinker further if you already have a working script (especially one that you understand and can modify further if needed).
I totally agree but I will keep your regex as reference, in the near future I will give it a try to decompose you regex as learning process but it looks rather very complex !
Another user came up with the following solution:
sed -E ':l;s/(\[[^]]*\]\()([^)#]*#[^)]*\))/\1\n\2/;Te;H;g;s/\n//;s/\n.*//;x;s/.*\n//;/^https?:/!{:h;s/^([^#]*#[^)]*)(%20|\.)([^)]*\))/\1-\3/;th;s/(#[^)]*\))/\L\1/;};tl;:e;H;z;x;s/\n//;'
Just as a little experiment, If you want to spend some time and give me a answer, what do you think? It’s a another way to achieve the same kind of results but they are significantly different. I know there a thousand ways to achieve the same results but I’m kinda curious how it looks from an experts eyes :).
Thanks again for your help and the time you took to write up a complex regex for my use case ! 👍
Hello :) Sorry to pin you, I just gave pandoc a try but it doesn’t work and I had to dig a bit further into the web to find out why !
Links to Headings with Spaces are not specified by CommonMark and each tool implement a different approach… Most replace space with hyphens other use URL encoding (%20). So even though pandoc looks awesome it doesn’t work for my use case (or did i miss something? Feel free to comment).
You can give it a try on https://pandoc.org/try/ with commonmark to gfm:
[Just a test](#Just a test)
[Just a link](https://mylink/%20with%20space.com)
[External link](Readme.md#JUST%20a%20test)
[Link with numbers](readme.md#1.3%20this%20is%20another%20test)
[Link with numbers](Another%20file%20to%20readme.md#1.3%20this%20is%20another%20test)
If you prefere a cli version:
pandoc --from=commonmark_x --to=gfm+gfm_auto_identifiers "/home/user/Documents/test.md" -o "pandoc_test.md"
Wow ! Thank you ! It did a rapid test on a test-file.md
[Just a test](#just-a-test)
[Just a link](https://mylink/%20with%20space.com)
[External link](readme.md#just-a-test)
[Link with numbers](readme.md#1-3-this-is-another-test)
[Link with numbers](Another%20file%20to%20readme.md#1-3-this-is-another-test)
Great job ! Thank you very much !!! I’m really impressed what someone with proper knowledge can do ! However, I really do not want to mess around with your regex… This will only call for disaster xD ! I will keep preciously your regex and annotated file in my knowledge base, I’m sure some time in the future I will come back to it and try to break it down as learning process.
Thank you very much !!! 👍
Yeah, I had a Android application installed recently which strangely enough tried to reach my vaultwarden DNS? That sounded sketchy AF and just blocked it, removed it and cleaned every trace of it…
I don’t really have a technical reason, but I do only named volumes to keep things clear and tidy, specially compose files with databases.
When I do a backup I run a script that saves each volumes/database/compose files well organized in directories archived with tar.
In have this structure in my home directory: /home/user/docker/application_name/docker-compose.yaml and it only contains the docker-compose.yml file (some times .env/Docker file).
I dunno if this is the most efficient way or even the best way to do things :/ but It also helps me to keep everything separate between all the necessary config files and the actual files (like movie files on Jellyfin) and it seems easier to switch over If I only need one part and not the other (uhhr sorry for my badly worded English, I hope it makes sense).
Other than that I also like to tinker arround and learn things :) Adding complexity gives me some kind of challenge? XD
Thank you very much for taking your time and trying to help me with comments and all !
you need a full featured markdown parser for this.
Do you mean something like pandoc? Someone pointed me to it and it seems it can covert to GitHub-Flavored Markdown ! Thanks for the pointer will give it a try to see how it works out with my actual script :)
Sorry for the very late response !! Here is the working bash script another user helped me put together:
#! /bin/bash
files="/home/USER/projects/test.md"
mdlinks="$(grep -Po ']\((?!https).*\)' "$files")"
mdlinks2="$(grep -Po '#.*' <<<$mdlinks)"
while IFS= read -r line; do
#Converts 1.2 to 1-2 (For a third level heading needs to add a supplementary [0-9])
dashlink="$(echo "$line" | sed -r 's|(.+[0-9]+)\.([0-9]+.+\))|\1-\2|')"
sed -i "s/$line/${dashlink}/" "$files"
#Puts everything to lowercase after a hashtag
lowercaselink="$(echo "$dashlink" | sed -r 's|#.+\)|\L&|')"
sed -i "s/$dashlink/${lowercaselink}/" "$files"
#Removes spaces (%20) from markdown links after a hashtag
spacelink="$(echo "$lowercaselink" | sed 's|%20|-|g')"
sed -i "s/$lowercaselink/${spacelink}/" "$files"
done <<<"$mdlinks2"
Hello :) Sorry for the very late response !
Effectively your regex is very close as a one line, I’m pretty impress ! :0 However I missed to mention something In my post (I only though about it after working on it with another user in the comments…). There a 2 things missing on your beautiful and complex regex:
FROM
---------------
[Link with numbers](readme.md#1.3%20this%20is%20another%20test)
TO
---------------
[Link with numbers](readme.md#1-3-this-is-another-test)
FROM
---------------
[Link with numbers](Another%20file%20to%20readme.md#1.3%20this%20is%20another%20test.md)
TO
---------------
[Link with numbers](Another%20file%20to%20readme.md#1-3-this-is-another-test.md)
Sorry for the trouble I wasn’t aware of all the GitHub-Flavored Markdown syntax :/. I got a a very cool working script that works perfectly with another user but If you want to modify your regex and try to solve the issue in pure regex feel free :) I’m very curious how It could look like (god regex is so obscure and at the same time it has some beauty in it !)
#! /bin/bash
files="/home/USER/projects/test.md"
mdlinks="$(grep -Po ']\((?!https).*\)' "$files")"
mdlinks2="$(grep -Po '#.*' <<<$mdlinks)"
while IFS= read -r line; do
#Converts 1.2 to 1-2 (For a third level heading needs to add a supplementary [0-9])
dashlink="$(echo "$line" | sed -r 's|(.+[0-9]+)\.([0-9]+.+\))|\1-\2|')"
sed -i "s/$line/${dashlink}/" "$files"
#Puts everything to lowercase after a hashtag
lowercaselink="$(echo "$dashlink" | sed -r 's|#.+\)|\L&|')"
sed -i "s/$dashlink/${lowercaselink}/" "$files"
#Removes spaces (%20) from markdown links after a hashtag
spacelink="$(echo "$lowercaselink" | sed 's|%20|-|g')"
sed -i "s/$lowercaselink/${spacelink}/" "$files"
done <<<"$mdlinks2"
Sorry for the late response… I was busy with another user :S My English is so bad I’m not able to response to every one at the same time… Whatever…
I tried your pearl regex substitution and effectively it does what I ask from my post, so thank you very much for your help ! However, I missed a few use cases were your regex breaks… But that’s on me, your command works as expected !!!
[Link with numbers](Another%20Markdown%20file.md#1.3%20this%20is%20another%20test.md)
The part before the hashtag need to keeps it’s original form (even with %20) because it links to a markdown file directly and not a header (Hope it’s comprehensible?). It took me a lot of time with another user and we came to a wrapped up script that does everything:
#! /bin/bash
files="/home/USER/projects/test.md"
mdlinks="$(grep -Po ']\((?!https).*\)' "$files")"
mdlinks2="$(grep -Po '#.*' <<<$mdlinks)"
while IFS= read -r line; do
#Converts 1.2 to 1-2 (For a third level heading needs to add a supplementary [0-9])
dashlink="$(echo "$line" | sed -r 's|(.+[0-9]+)\.([0-9]+.+\))|\1-\2|')"
sed -i "s/$line/${dashlink}/" "$files"
#Puts everything to lowercase after a hashtag
lowercaselink="$(echo "$dashlink" | sed -r 's|#.+\)|\L&|')"
sed -i "s/$dashlink/${lowercaselink}/" "$files"
#Removes spaces (%20) from markdown links after a hashtag
spacelink="$(echo "$lowercaselink" | sed 's|%20|-|g')"
sed -i "s/$lowercaselink/${spacelink}/" "$files"
done <<<"$mdlinks2"
If you are motivated you can still improve your regex If you want :) I’m kinda curious If it’s possible with a one-liner ! Thank again for your help and sorry for the late response !!
Yeah probably bare bone regex was a mistake however a friendly user gave me a step by step guide on how to achieve my goal:
#! /bin/bash
files="/home/USER/projects/test.md"
mdlinks="$(grep -Po ']\((?!https).*\)' "$files")"
mdlinks2="$(grep -Po '#.*' <<<$mdlinks)"
while IFS= read -r line; do
#Converts 1.2 to 1-2 (For a third level heading needs to add a supplementary [0-9])
dashlink="$(echo "$line" | sed -r 's|(.+[0-9]+)\.([0-9]+.+\))|\1-\2|')"
sed -i "s/$line/${dashlink}/" "$files"
#Puts everything to lowercase after a hashtag
lowercaselink="$(echo "$dashlink" | sed -r 's|#.+\)|\L&|')"
sed -i "s/$dashlink/${lowercaselink}/" "$files"
#Removes spaces (%20) from markdown links after a hashtag
spacelink="$(echo "$lowercaselink" | sed 's|%20|-|g')"
sed -i "s/$lowercaselink/${spacelink}/" "$files"
done <<<"$mdlinks2"
If you know a better way to achieve similar results I’m very open for every new lead and learn something new !
Thanks for the pointer I wasn’t aware pandoc was able to do that :/ It seems It can convert to Github-Flavored Markdown !! I have to give it a try :) Still I learned a lot from another user about regex/sed and Pearl :) !
Hello :) Sorry for the late response !!! I was busy working it out with another user ! However out of curiosity gave your sed regex a try, but there seems a missing ( somewhere ! I tried to fix the issue but your regex is way over my capabilities ! If you are sed/regex fanatic a want to give it another try feel free :). Right now I found a solution with another user that works great here’s the script in question if you are interested:
#! /bin/bash
files="/home/USER/projects/test.md"
mdlinks="$(grep -Po ']\((?!https).*\)' "$files")"
mdlinks2="$(grep -Po '#.*' <<<$mdlinks)"
while IFS= read -r line; do
#Converts 1.2 to 1-2 (For a third level heading needs to add a supplementary [0-9])
dashlink="$(echo "$line" | sed -r 's|(.+[0-9]+)\.([0-9]+.+\))|\1-\2|')"
sed -i "s/$line/${dashlink}/" "$files"
#Puts everything to lowercase after a hashtag
lowercaselink="$(echo "$dashlink" | sed -r 's|#.+\)|\L&|')"
sed -i "s/$dashlink/${lowercaselink}/" "$files"
#Replace spaces (%20) from markdown links to - after a hashtag
spacelink="$(echo "$lowercaselink" | sed 's|%20|-|g')"
sed -i "s/$lowercaselink/${spacelink}/" "$files"
done <<<"$mdlinks2"
It’s not very elegant but it does the job… While working on it with another very friendly user I came across other thing I haven’t though of like:
[]()[](Another%20File.md)[](Another%20File.md)Well I think that bare bone sed/regex wasn’t the right tool, but in a bash script it does exactly what I’m expecting :)
Thanks for your help and pointers !
Hello :) I promise this is the last time I will bother you (I know what you are going to say :P) ! If it’s not to much could you give me just a few hints on how I could improve a bit the final script?
#! /bin/bash
files="/home/USER/projects/test.md"
mdlinks="$(grep -Po ']\((?!https).*\)' "$files")"
mdlinks2="$(grep -Po '#.*' <<<$mdlinks)"
while IFS= read -r line; do
#Converts 1.2 to 1-2 (For a third level heading needs to add a supplementary [0-9])
dashlink="$(echo "$line" | sed -r 's|(.+[0-9]+)\.([0-9]+.+\))|\1-\2|')"
sed -i "s/$line/${dashlink}/" "$files"
#Puts everything to lowercase after a hashtag
lowercaselink="$(echo "$dashlink" | sed -r 's|#.+\)|\L&|')"
sed -i "s/$dashlink/${lowercaselink}/" "$files"
#Removes spaces (%20) from markdown links after a hashtag
spacelink="$(echo "$lowercaselink" | sed 's|%20|-|g')"
sed -i "s/$lowercaselink/${spacelink}/" "$files"
done <<<"$mdlinks2"
This works perfectly en fulfills all my needs (thanks !!) ! However I’m not very fond of the variable string manipulation ($mdlinks2), if you have some tips without spoiling to much, would be great, otherwise it’s okay, it works exactly how I have imagined it and ticks all use cases. Also If you could give some pointer for an overall improvement or if you see something that could potentially create some strange loop or looks off feel free to comment in your spare time :).
Another question which has nothing to do with the post and gets a bit off topic… You gave me the right push I needed and I saw the power and usefulness of proper knowledge with sed/bash/Pearl. It’s time I finally learn a scripting language ! I want to hear your opinion on what tools would you recommend? Most people would say Python for beginners but I heard so much good things about Pearl (Exiftool is a good example of how powerful Pearl can be) but the syntax scares me out a little bit compared to Python.
Any good book material you have in mind for a beginner?
Thanks again for everything !!!
First, thanks again for sharing your knowledge with me I really appreciate the time/effort you took to write all of this. I know those are a lot of thank you :/ but I’m really grateful for all of this, this is very valuable information I will keep in my knowledge base. It’s really time I learn proper bash/python/Pearl? scripting with all those tools (grep/sed/regex).
Second, YOU MISSED A DAMNED parentheses you fool xD ! mdlinks="$(grep -Po ']\((?!https).*\)' ~/mkdn)" Took me some time to figured it out with a very non informative error bashscript.sh: line 8: unexpected EOF while looking for matching "' but as expected it works !
From
-------
[Just a test](#Just%20a%20test.md)
[Just a link](https://mylink/%20with%20space.com)
%20
To
-------
[Just a test](#Just-a-test.md)
[Just a link](https://mylink/%20with%20space.com)
%20
Next to show you my appreciation and not to take everything for granted and being spoon feed for everything, I tried to find a solution myself for something else, I will try to explain the best I can how I solved it.
From
-------
[Just a test](Another%20markdown%20file.md#Hello%20World)
To
-------
[Just a test](Another%20markdown%20file.md#hello-world)
The part before the hashtag needs to keep it’s initial form (it links to the original markdown file). So, because just playing around with Pearl and regex (which doesn’t end well doing this blindly without the proper knowledge) I did some simple string manipulation. It’s not very elegant but does the trick, thankfully to your well written breakdown.
#! /bin/bash
mdlinks="$(grep -Po ']\((?!https).*\)' "/home/dany/newtest.md")"
echo $mdlinks
mdlinks2="$(grep -Po '#.*' <<<$mdlinks)"
echo $mdlinks2
while IFS= read -r line; do
dashlink="$(echo "$line" | sed 's|%20|-|g')"
sed -i "s/$line/${dashlink}/" "/home/dany/newtest.md"
done <<<"$mdlinks2"
Yes, not very elegant but It’s the best I could do currently :/ However, I still got a YES effect :P
To answer your question:
Quick question as I’m working on this, in the new link example, is the BDMV and other capitalized text in this link supposed to be converted to lowercase, or to remain uppercase?
As you can see in my string manipulation above, the part before the # needs to keep it’s original form :) (Sorry wasn’t aware of this before working with the original files) I solved it with some string manipulation as shown above.
I’m a bit tired from all this searching/trail&error, tomorrow I will try to wrap everything up and answer your post below :) ! Also, I need to clean up the mess I made in my home directory xD.
Thanks again for your help ! Have a good night/day !
Hello !!!
Sorry for the very late response had something else to do. I will read everything carefully and response to every post :) I also thought about it over night and I think that sed and and regex wasn’t the best option here (as other have mentioned it).
I think a python script or bash (as you have mentioned it a bit later ) would be a better way. I’m sorry that I put you through all of this… wrong tool for the job :s.
Heeey ! Take that back !! We aren’t crazy, just different. Thats cool, just enjoy what ever you think is good for your mental health !
Just like some people love healthy meals and other can’t live without processed food 😏
Yeah :) piped self-hosting !
However, we have to wait until they merge the poToken and visitor data issue. Something invidious has already implemented.
Still it’s very strange you have to restart your container every hour :/ I’m sure I’m to stupid to understand the backend/code to make any sense out of it, but piped instances never had to restart anything.