~dmbaturin/soupault

4

Incompatible changes for the 2.0 release

Details
Message ID
<74cd3e13-2c26-8431-a69f-91b5bec9f899@baturin.org>
DKIM signature
pass
Download raw message
I've made a point to break compatibility as rarely as possible, but I
fear soon it will be time for a 2.0 release.

Some design decisions didn't pass the test of time, and changing the
config syntax and/or behaviour can make some thing more logical and
flexible.
If we are going to do it, it's a good idea to make as many incompatible
changes at once as possible so that people will only need one big migration.
Small incompatible changes in every release is a horrible practice.

My plan how to make that migration simpler: write a script for
converting old configs to the new format. That can even be a client-side
web app where you can paste your config.

However, we need to identify all things we want to break, and I need
your input.

1. Get rid of built-in index fields

Initially there was this built-in "content model":

[settings]
  title_selector =
  author_selector =
  date_selector =
  index_date_format =

By now, custom fields are _more_ functional than most built-ins because
they can extract attribute content:

[index.custom_fields]
  foo = { selector = "#foo", extract_attribute = "bar"}

I see no sensible way to extend built-in options to support attribute
extraction. Other than adding new prefixed options under settings...

I also think index_data_format should also be generalized.
Every field can have options like sort_order = <"lexicographic" |
"numeric" | "date"> and date_format. Better a list of date formats to try.


2. Get rid of default_template

Now we support multiple page templates. In this light, default_template
and content_selector become fallback options.

Something like this seems easier to read and edit:

[templates.default]
  file = "templates/main.html"
  content_selector = "body"

[templates.funny]
  file = "templates/funny.html"
  section = "fun"
  content_selector = "main"


3. Your ideas?

If you could break anything and get away with it, what would you do?
Details
Message ID
<a6b3fa0e-b91d-ed22-53a1-cfdc55196e9d@aoirthoir.com>
In-Reply-To
<74cd3e13-2c26-8431-a69f-91b5bec9f899@baturin.org> (view parent)
DKIM signature
pass
Download raw message
I agree here are my thoughts section by section...

==========
[index.custom_fields]
   foo = { selector = "#foo", extract_attribute = "bar"}
==========

Allow custom fields to include things besides/selectors attributes only 
such as:

inner_html
filename
luacode return value or luacode function return value (see my 
do-code.lua plugin).
--- this would allow us to create libraries of variables...but the user 
could choose
--- which variables to use in which plugins without writing a plugin 
themselves...
dates/times
calculations etc...

Further allow custom_fields to be used in any plugin not just indexes... 
i suppose
this means something like fields.name in lua.. fields is preferable to a 
long name like
custom_fields.. and continues the tradition of config.name.. (maybe 
custom_fields
already can be used anywhere..as yet im not up enough on the index 
features of
soupault...ive read them but not tested...

==============
along with the idea of custom_fields, widget-defaults...
==============

For instance my block-maker.lua plugin requires:
wrapper = 'block level element' defaults to div in the plugin
class      = 'class to be applied to wrapper' defaults to class of 
element being wrapped...

The soupault.conf for this looks like this:

[widgets.espanol]
   widget    = 'block-maker'
   start     = 'Espanol'

[widgets.sonidos]
   widget    = 'block-maker'
   start     = 'Sonidos'

[widgets.ingles]
   widget    = 'block-maker'
   start     = 'Ingles'

This code is fine because they are all going to go into a div so no need 
to specify wrapper=
but if i want them to go into something else then i have to do this:

[widgets.espanol]
   widget    = 'block-maker'
   start     = 'Espanol'
   wrapper = 'article'

[widgets.sonidos]
   widget    = 'block-maker'
   start     = 'Sonidos'
   wrapper = 'article'

[widgets.ingles]
   widget    = 'block-maker'
   start     = 'Ingles'
   wrapper = 'article'

So now if we have widget-defaults we get this...

[widget_defaults]
   wrapper = 'article'

[widgets.espanol]
   widget    = 'block-maker'
   start     = 'Espanol'

[widgets.sonidos]
   widget    = 'block-maker'
   start     = 'Sonidos'

[widgets.ingles]
   widget    = 'block-maker'
   start     = 'Ingles'
   wrapper = 'div'

All the widgets use the default wrapper except the last one which 
overrides back div..
We could even do this:

[widget_defaults]
   wrapper = 'article'

[widgets.espanol]
   widget    = 'block-maker'
   start     = 'Espanol'

[widgets.sonidos]
   widget    = 'block-maker'
   start     = 'Sonidos'

[widgets.ingles]
   widget    = 'block-maker'
   start     = 'Ingles'
   wrapper = '//PLUGIN-DEFAULT//'

The last is some clue that just gives an idea that whatever the plugin 
default is, use that.
This default scenario is only needed when you want to override a 
widget-default that was set.

=========================
sort order is very important.. agreed
=========================

=========================
2. Get rid of default_template
=========================

[templates.default]
   file = "templates/main.html"
   content_selector = "body"

[templates.funny]
   file = "templates/funny.html"
   section = "fun"
   content_selector = "main"

So with something like this i would still have templates_dir so that you 
could just say

file="main.html"

Now that you have the ability to do different templates take a look at 
my tree-template-include.lua..
Consider adding that as a built in option... so we could get:

[templates.default]
   file = 'main.html'
   tree = 'true/false' (true means.. go further up the tree to find it 
if it exists.. )

http://localhost/add1tocobol-lua/#file-tree-search-explained

This explains it...

This way we have the ability to drop into a directory tree the template 
we want or use a setting in soupault.conf
Details
Message ID
<028624d9-f77e-6637-4cb3-7ad316f4fb89@aoirthoir.com>
In-Reply-To
<a6b3fa0e-b91d-ed22-53a1-cfdc55196e9d@aoirthoir.com> (view parent)
DKIM signature
pass
Download raw message
I decided to go through your plugins so i could see how you were 
handling things... for instance i thought you had a plugin to convert to 
html_entities but when i wa writing all that code these past couple 
weeks i could not remember what it was. i thought it was a built in 
widget. in any case i wrote my own with my library so i wanna share and 
compare and explain in case you wanna use these kind of ideas in the main...

This is your code from escape-html.lua:

========================
selector = config["selector"]
if not selector then
   Plugin.fail("Missing required option \"selector\"")
end

function escape_html(element)
   content = HTML.inner_html(element)
   -- HTML.create_text escapes HTML special characters
   content = HTML.create_text(content)
   HTML.replace_content(element, content)
end

elements = HTML.select(page, selector)

if not elements then
   Plugin.exit("No elements found, nothing to do")
end

local index = 1
while elements[index] do
   escape_html(elements[index])
   index = index + 1
end
========================

This is my code from pre-me.lua that does the same thing:

========================
Plugin.require_version('1.8')
dofile('plugins/add1tocobol/add1tocobol.lua')

ADD1.default_selectors('pre')

function pre_me()
   data =  HTML.inner_html(element)
   ADD1.process_element('replace_content', element, data, 'raw')
end

ADD1.loop_selectors('pre_me()')
========================

So my thoughts are that some of these concepts could be incorporated 
directly into the core including the selectors table, the 
loop_selectors(), the loop_elements(), head, body and so on, which i do 
this way:

     body =  HTML.select_one(page, 'body')
     head =  HTML.select_one(page, 'head')

In addition if we had access to all settings soupault.conf...or the 
toml..for  our plugins, the ability to read but not change... that would 
be handy as well.. more variables related to the page.. is good..
Details
Message ID
<407968f1-9c67-3984-e5de-413c5778f4f4@aoirthoir.com>
In-Reply-To
<a6b3fa0e-b91d-ed22-53a1-cfdc55196e9d@aoirthoir.com> (view parent)
DKIM signature
pass
Download raw message
Log.warning()
Log.info()
Plugin.fail()
Plugin.exit()
Plugin.require_version()

automatically include page_file and plugin_file.. so we know where it 
was called from...

Let me know if these ideas are cumbersome since I am posting so many. 
Any you want you can discard of course...
Details
Message ID
<51cbf5c2-7cc0-10ed-bbea-375afc27ca47@aoirthoir.com>
In-Reply-To
<a6b3fa0e-b91d-ed22-53a1-cfdc55196e9d@aoirthoir.com> (view parent)
DKIM signature
pass
Download raw message
So right now I am running this error message code:

success, data = ADD1.get_file(filepath)
if success == 'SUCCESS' then
   ADD1.process_element('replace_element', element, data, raw)
else
   Log.warning(
     ' page_file:'..page_file..
     ' plugin:include-tag.lua'..
     ' error:'..success..
     ' filepath:'..filepath)
end

Giving me this result:

aoirthoir@AoirthoirInspireWone ~/R/T/s/cards> soupault

[WARNING] page_file:site/html.html plugin:include-tag.lua error:NOT A 
FILENAME filepath:
[WARNING] page_file:site/html.html plugin:include-tag.lua error:FILE NOT 
FOUND filepath:joey.txt

aoirthoir@AoirthoirInspireWone ~/R/T/s/cards>
Export thread (mbox)