Philip-san,
I have a question about site-lisp sub directory handling.
In my case, I have:
~/.emacs.d/site-lisp
~/.emacs.d/site-lisp/progmodes
and set `site-lisp-directory' to "site-lisp" (default).
In this case, `site-lisp-prepare' tries to traverse "progmodes"
directory, but fails to add its autoloads.
If I set `site-lisp-directory' to
(list "~/.emacs.d/site-lisp" "~/.emacs.d/site-lisp/progmodes")
, two ".auto-site.el"s are generated as expected, but
`site-lisp-prepare' does redundant job according to *Messages* log.
I think we do not need to add "progmodes" to `site-lisp-directory', and
may need to set autoload-file for each directory in `site-lisp-prepare'
when traverses sub directories.
--
TAKAHASHI Yoshio <yfb02119@nifty.com>
TAKAHASHI Yoshio <yfb02119@nifty.com> writes:
> Philip-san,
Hi,
> I have a question about site-lisp sub directory handling.>> In my case, I have:> ~/.emacs.d/site-lisp> ~/.emacs.d/site-lisp/progmodes> and set `site-lisp-directory' to "site-lisp" (default).> In this case, `site-lisp-prepare' tries to traverse "progmodes"> directory, but fails to add its autoloads.
That sounds like a reasonable feature, but I wouldn't want to hard-code
"progmodes". How does this patch look like to you:
Philip-san,
Thank you for your reply, I try to explain cases in my side, my
expectation, and experience.
Philip Kaludercic <philipk@posteo.net> writes:
>> In my case, I have:>> ~/.emacs.d/site-lisp>> ~/.emacs.d/site-lisp/progmodes> That sounds like a reasonable feature, but I wouldn't want to hard-code> "progmodes". How does this patch look like to you:
I do not want to hard-code subdirectories' name, too.
>> I think we do not need to add "progmodes" to `site-lisp-directory', and>> may need to set autoload-file for each directory in `site-lisp-prepare'>> when traverses sub directories.> Why do you think we need separate autoload-files? More autoload files> means more files have to be opened at startup, which slows Emacs down> (albeit just slightly, but it also depends on your file system).
I have not strong idea that we need to have multiple autoload files.
But, my expectation is to generate autoload-file(s) from
site-lisp/**/*.el when I set `site-lisp-directory' to site-lisp/.
I want to understand expected behavior of site-lisp 0.1.2.
When I set `site-lisp-directory' to ~/.emacs.d/sitelisp (default),
.auto-site.el only have autoloads of site-lisp/*.el, and does not have
autoloads of site-lisp/*/*.el. Is this expected?
From my rough understanding, `loaddefs-generate' generates autoload-file
for site-lisp, then read subdirectory. Generated autoload-file is newer
than files in subdirectory, no autoloads are collected from subdirectory.
I tried below code. It assigns autoload-file for each directory, and
autoloads are collected into .auto-site.el in each directory.
(defun site-lisp-prepare (dir)
"Byte-compile, prepare autoloads and note each directory in DIR.
If DIR is a list, the function will be applied to every element
of the list."
(interactive "DPrepare: ")
(if (listp dir)
(mapc #'site-lisp-prepare dir)
(let ((backup-inhibited t))
(dolist (dir (cons dir (directory-files dir t "^[^.]")))
(when (file-directory-p dir)
(let ((autoload-file (expand-file-name site-lisp-autoload-file dir)))
(message "Site-lisp: %s" dir)
(add-to-list 'load-path dir)
(site-lisp-generate-autoloads dir autoload-file)
(load autoload-file nil t))))
(byte-recompile-directory dir))))
--
TAKAHASHI Yoshio <yfb02119@nifty.com>
TAKAHASHI Yoshio <yfb02119@nifty.com> writes:
> Philip-san,>> Thank you for your reply, I try to explain cases in my side, my> expectation, and experience.>> Philip Kaludercic <philipk@posteo.net> writes:>>>> In my case, I have:>>> ~/.emacs.d/site-lisp>>> ~/.emacs.d/site-lisp/progmodes>>> That sounds like a reasonable feature, but I wouldn't want to hard-code>> "progmodes". How does this patch look like to you:>> I do not want to hard-code subdirectories' name, too.>>>> I think we do not need to add "progmodes" to `site-lisp-directory', and>>> may need to set autoload-file for each directory in `site-lisp-prepare'>>> when traverses sub directories.>> Why do you think we need separate autoload-files? More autoload files>> means more files have to be opened at startup, which slows Emacs down>> (albeit just slightly, but it also depends on your file system).>> I have not strong idea that we need to have multiple autoload files.> But, my expectation is to generate autoload-file(s) from> site-lisp/**/*.el when I set `site-lisp-directory' to site-lisp/.
No, that is not what the package offers. All it currently does is to
load ./*.el and ./**/&.el in `site-lisp-directory', but if you want this
functionality, we can add it with a user option.
> I want to understand expected behavior of site-lisp 0.1.2.> When I set `site-lisp-directory' to ~/.emacs.d/sitelisp (default),> .auto-site.el only have autoloads of site-lisp/*.el, and does not have> autoloads of site-lisp/*/*.el. Is this expected?
No, those files _should_ also be loaded.
> From my rough understanding, `loaddefs-generate' generates autoload-file> for site-lisp, then read subdirectory. Generated autoload-file is newer> than files in subdirectory, no autoloads are collected from subdirectory.
Yes, that is what it should do.
> I tried below code. It assigns autoload-file for each directory, and> autoloads are collected into .auto-site.el in each directory.>> (defun site-lisp-prepare (dir)> "Byte-compile, prepare autoloads and note each directory in DIR.> If DIR is a list, the function will be applied to every element> of the list."> (interactive "DPrepare: ")> (if (listp dir)> (mapc #'site-lisp-prepare dir)> (let ((backup-inhibited t))> (dolist (dir (cons dir (directory-files dir t "^[^.]")))> (when (file-directory-p dir)> (let ((autoload-file (expand-file-name site-lisp-autoload-file dir)))> (message "Site-lisp: %s" dir)> (add-to-list 'load-path dir)> (site-lisp-generate-autoloads dir autoload-file)> (load autoload-file nil t))))> (byte-recompile-directory dir))))
While possible, I don't get what advantage to doing this.
Philip-san,
Philip Kaludercic <philipk@posteo.net> writes:
>> But, my expectation is to generate autoload-file(s) from>> site-lisp/**/*.el when I set `site-lisp-directory' to site-lisp/.> No, that is not what the package offers. All it currently does is to> load ./*.el and ./**/&.el in `site-lisp-directory',>> and does not have>> autoloads of site-lisp/*/*.el. Is this expected?> No, those files _should_ also be loaded.>> From my rough understanding, `loaddefs-generate' generates autoload-file>> for site-lisp, then read subdirectory. Generated autoload-file is newer>> than files in subdirectory, no autoloads are collected from subdirectory.> Yes, that is what it should do.
I'm lost. `site-lisp-prepare' sets load-path and loads autoload-files.
If we don't have autoload-file for subdirectory *.el, we can not call
functions in those files without explicit require or load the file.
> While possible, I don't get what advantage to doing this.
It generates .auto-site.el in subdirectory, I can use/call functions in
subdirectores el-files without load those el-files explicitly.
--
TAKAHASHI Yoshio <yfb02119@nifty.com>
TAKAHASHI Yoshio <yfb02119@nifty.com> writes:
> Philip-san,>> Philip Kaludercic <philipk@posteo.net> writes:>>>> But, my expectation is to generate autoload-file(s) from>>> site-lisp/**/*.el when I set `site-lisp-directory' to site-lisp/.>> No, that is not what the package offers. All it currently does is to>> load ./*.el and ./**/&.el in `site-lisp-directory',>>>> and does not have>>> autoloads of site-lisp/*/*.el. Is this expected?>> No, those files _should_ also be loaded.>>>> From my rough understanding, `loaddefs-generate' generates autoload-file>>> for site-lisp, then read subdirectory. Generated autoload-file is newer>>> than files in subdirectory, no autoloads are collected from subdirectory.>> Yes, that is what it should do.>> I'm lost. `site-lisp-prepare' sets load-path and loads autoload-files.> If we don't have autoload-file for subdirectory *.el, we can not call> functions in those files without explicit require or load the file.
An autoload-file doesn't have to be restricted to fines in a directory,
it can also contain the information for files anywhere else on the file
system, in our case in recursive subdirectories.
>> While possible, I don't get what advantage to doing this.>> It generates .auto-site.el in subdirectory, I can use/call functions in> subdirectores el-files without load those el-files explicitly.
I get that, that is the point of autoloads, what I wanted to know is if
you had a use-case for separating autoload files for each directory,
instead of gathering them all in the "root" directory.
--
Philip Kaludercic
Philip-san,
Philip Kaludercic <philipk@posteo.net> writes:
>>> While possible, I don't get what advantage to doing this.>> It generates .auto-site.el in subdirectory, I can use/call functions in>> subdirectores el-files without load those el-files explicitly.> I get that, that is the point of autoloads, what I wanted to know is if> you had a use-case for separating autoload files for each directory,> instead of gathering them all in the "root" directory.
I worry we have mis-communication.
If root .auto-site.el contains all autoloads in site-lisp/**/*.el, it is
good thing.
In my environment, site-lisp 0.1.2 does not gather autoloads in
subdirectories into root .auto-site.el. I need to load or require
el-file in subdirectory explicitly before using commands.
According to `loaddefs-geenrate', simplest way to gather auto-loads
seems to have .auto-site.el in each directory, then I experimentally
modified `site-lisp-prepare'.
".auto-site.el in each directory" is the side-effect of my modification.
--
TAKAHASHI Yoshio <yfb02119@nifty.com>
TAKAHASHI Yoshio <yfb02119@nifty.com> writes:
> Philip-san,>> Philip Kaludercic <philipk@posteo.net> writes:>>>>> While possible, I don't get what advantage to doing this.>>> It generates .auto-site.el in subdirectory, I can use/call functions in>>> subdirectores el-files without load those el-files explicitly.>> I get that, that is the point of autoloads, what I wanted to know is if>> you had a use-case for separating autoload files for each directory,>> instead of gathering them all in the "root" directory.>> I worry we have mis-communication.>> If root .auto-site.el contains all autoloads in site-lisp/**/*.el, it is> good thing.>> In my environment, site-lisp 0.1.2 does not gather autoloads in> subdirectories into root .auto-site.el. I need to load or require> el-file in subdirectory explicitly before using commands.
Our miscommunication appears to be that I didn't make it clear, that the
behaviour I was describing was a suggestion on how to fix the issue. I
have now pushed a commit to the repository (e2dca35d) that should
implement the feature -- but haven't tested it yet, so please say if you
notice any issues.
If not, I can prepare a new release.
> According to `loaddefs-geenrate', simplest way to gather auto-loads> seems to have .auto-site.el in each directory, then I experimentally> modified `site-lisp-prepare'.>> ".auto-site.el in each directory" is the side-effect of my modification.
Philip-san,
Philip Kaludercic <philipk@posteo.net> writes:
> I> have now pushed a commit to the repository (e2dca35d) that should> implement the feature -- but haven't tested it yet, so please say if you> notice any issues.
Unfortunately, it does not work as expected.
I set `site-lisp-fixed-subdirectories' to my site-lisp subdirectores,
and set `site-lisp-collect-recursivly' to `t' results in
Lisp nesting exceeds `max-lisp-eval-depth': 1601
. This is because `site-lisp-prepare' calls itself with
".emacs.d/site-lisp" as its argument repeatedly.
When set `site-lisp-collect-recursivly' to `nil', I do not find
autoloads of subdirectories in root .auto-site.el.
--
TAKAHASHI Yoshio <yfb02119@nifty.com>
TAKAHASHI Yoshio <yfb02119@nifty.com> writes:
> Philip-san,>> Philip Kaludercic <philipk@posteo.net> writes:>>> I>> have now pushed a commit to the repository (e2dca35d) that should>> implement the feature -- but haven't tested it yet, so please say if>> you>> notice any issues.>> Unfortunately, it does not work as expected.>> I set `site-lisp-fixed-subdirectories' to my site-lisp subdirectores,> and set `site-lisp-collect-recursivly' to `t' results in> Lisp nesting exceeds `max-lisp-eval-depth': 1601> . This is because `site-lisp-prepare' calls itself with> ".emacs.d/site-lisp" as its argument repeatedly.>> When set `site-lisp-collect-recursivly' to `nil', I do not find> autoloads of subdirectories in root .auto-site.el.
Sorry about that, just now I notice that my changes didn't make any
sense (but as I said, I hadn't tested them). The newest commit should
resolve the situation, at least it looked like it from some quick
testing.
--
Philip Kaludercic on peregrine
Philip Kaludercic <philipk@posteo.net> writes:
> The newest commit should> resolve the situation, at least it looked like it from some quick> testing.
Umm, 04eb658b6449b8a6a4137b954b35b66006dca515 does not work.
I quickly finds out two problems.
In line 51, `site-lisp-autoload-file' is set, check the value in Line 89
is useless.
`directory-files' in line 91 is called `t' as the second parameter, it
returns full path, `member' never finds full-path value of DIR in
`site-lis-fixed-subdirectories'.
--
TAKAHASHI Yoshio <yfb02119@nifty.com>
TAKAHASHI Yoshio <yfb02119@nifty.com> writes:
> Philip Kaludercic <philipk@posteo.net> writes:>>> The newest commit should>> resolve the situation, at least it looked like it from some quick>> testing.>> Umm, 04eb658b6449b8a6a4137b954b35b66006dca515 does not work.>> I quickly finds out two problems.>> In line 51, `site-lisp-autoload-file' is set, check the value in Line 89> is useless.
Good catch, the intention was to have a separate variable for the full
path, so that all autoload cookies get stored in that fixed position.
I have already pushed a commit addressing this.
> `directory-files' in line 91 is called `t' as the second parameter, it> returns full path, `member' never finds full-path value of DIR in> `site-lis-fixed-subdirectories'.
That's a different problem, right. I should (and now do) call
`file-name-nondirectory' before passing the file to member (I always get
mixed up with these functions, but this is not related to the problem at
hand).
--
Philip Kaludercic on peregrine
Philip Kaludercic <philipk@posteo.net> writes:
> I should (and now do) call> `file-name-nondirectory' before passing the file to member
0f66f9f81f8e0c8ac1b94b2b14f615f3073100d8 ends in infinite loop, when
`site-lisp-fixed-subdirectories' has subdirectories. Because we add
fixed-subdirectories again at line 91, and recursive call with the same
parameter at line 95.
91: (dolist (dir (cons dir (directory-files dir t "^[^.]")))
92: (when (file-directory-p dir)
93: (if (member (file-name-nondirectory dir)
94: site-lisp-fixed-subdirectories)
95: (site-lisp-prepare dir)
If we set `site-lisp-fixed-subdirectories' to nil, and set
`site-lisp-collect-recursivly' to t, emacs starts. But, .auto-site.el
does not contain autoloads in subdirectories.
Because .auto-site.el is newer than subdirectories/*.el.
I'll continue to use my experimental code shared in my privious mail.
--
TAKAHASHI Yoshio <yfb02119@nifty.com>
TAKAHASHI Yoshio <yfb02119@nifty.com> writes:
> Philip Kaludercic <philipk@posteo.net> writes:>>> I should (and now do) call>> `file-name-nondirectory' before passing the file to member>> 0f66f9f81f8e0c8ac1b94b2b14f615f3073100d8 ends in infinite loop, when> `site-lisp-fixed-subdirectories' has subdirectories. Because we add> fixed-subdirectories again at line 91, and recursive call with the same> parameter at line 95.>> 91: (dolist (dir (cons dir (directory-files dir t "^[^.]")))> 92: (when (file-directory-p dir)> 93: (if (member (file-name-nondirectory dir)> 94: site-lisp-fixed-subdirectories)> 95: (site-lisp-prepare dir)
I am inclined to just remove `site-lisp-fixed-subdirectories' at this
point.
>> If we set `site-lisp-fixed-subdirectories' to nil, and set> `site-lisp-collect-recursivly' to t, emacs starts. But, .auto-site.el> does not contain autoloads in subdirectories.> Because .auto-site.el is newer than subdirectories/*.el.
Right; In that case we could use temporary files to store the output
temporarily, and then merge it all together into a single file. Or we
could store the initial timestamp and always restore that?
> I'll continue to use my experimental code shared in my privious mail.
--
Philip Kaludercic on peregrine
Philip Kaludercic <philipk@posteo.net> writes:
>> Because .auto-site.el is newer than subdirectories/*.el.> Right; In that case we could use temporary files to store the output> temporarily, and then merge it all together into a single file. Or we> could store the initial timestamp and always restore that?
It may work, but the merge process is not obvious for me. Do we need to
modify paths in temporary .auto-site.el during merge? And, I wonder it
may be too complex to keep initial timestamp. We may need to pass the
timestamp to `site-lisp-generate-autoloads' (and change
`loaddefs-generate', `make-directory-autoloads', or
`update-directory-autoloads').
Currently, I can live with .auto-site.el in each subdirectory...
--
TAKAHASHI Yoshio <yfb02119@nifty.com>
TAKAHASHI Yoshio <yfb02119@nifty.com> writes:
> Philip Kaludercic <philipk@posteo.net> writes:>>>> Because .auto-site.el is newer than subdirectories/*.el.>> Right; In that case we could use temporary files to store the output>> temporarily, and then merge it all together into a single file. Or we>> could store the initial timestamp and always restore that?>> It may work, but the merge process is not obvious for me. Do we need to> modify paths in temporary .auto-site.el during merge?
The issue is that `site-lisp-autoload-file' is just a filename, not a
path, but it is interpreted as a relative filename, and therefore
generates a separate file for each directory (which I want to avoid).
> And, I wonder it> may be too complex to keep initial timestamp. We may need to pass the> timestamp to `site-lisp-generate-autoloads' (and change> `loaddefs-generate', `make-directory-autoloads', or> `update-directory-autoloads').
The idea is not that complicated, I haven't pushed the code yet because
I want to test it for once, but the idea is just to store the previous
mtime before recursing, and then resture it afterwards.
> Currently, I can live with .auto-site.el in each subdirectory...
It would be possible, but the downside is the additional startup cost of
finding and loading all the separate files, instead of doing that when
the site-lisp directory is being prepared. The only advantage is that
it would be simpler to implement, but I don't think by a lot.
--
Philip Kaludercic on icterid
Philip Kaludercic <philipk@posteo.net> writes:
> It would be possible, but the downside is the additional startup cost of> finding and loading all the separate files,
In my case, load-path length is more than 100 already. 26 from
"emacs/30.0.50/lisp/*", 70 from "~/.emacs.d/elpa/*". Add several pathes
to load-path may be negligible. What is your load-path length?
--
TAKAHASHI Yoshio <yfb02119@nifty.com>
TAKAHASHI Yoshio <yfb02119@nifty.com> writes:
> Philip Kaludercic <philipk@posteo.net> writes:>>> It would be possible, but the downside is the additional startup>> cost of>> finding and loading all the separate files,>> In my case, load-path length is more than 100 already. 26 from> "emacs/30.0.50/lisp/*", 70 from "~/.emacs.d/elpa/*". Add several pathes> to load-path may be negligible. What is your load-path length?
The length of `load-path' itself is not the issue here (mine is
currently coincidentally exactly 100), but the number of files one would
have to access, parse and load during startup time. If everything is
stored in a single file, then only one file has to be opened, which is
preferable to traversing the entire directory for an unknown number of
files. That is also the same reason why the `package-quickstart' option
exists.
--
Philip Kaludercic on peregrine
Philip Kaludercic <philipk@posteo.net> writes:
> The length of `load-path' itself is not the issue here
Yes. What I want to share is the fact that every elpa directories (70,
in my case) has its autoload file.
> If everything is> stored in a single file, then only one file has to be opened, which is> preferable to traversing the entire directory for an unknown number of> files.
Thank you, I did not know `package-quickstart'.
8a7f14a45e62b3dd4672cf3094baeb58716df62d.
- `file-attributes' returns nil on non-existing file, in the case there
is no .auto-site.el. Its our first execution after install
`site-lisp.el'. `set-file-times' uses current time when the parameter
is `nil'. In this case, .auto-site.el contains autoloads in
site-lisp/subdirectory/*.el, but does not contain any autoloads in
.emacs.d/site-lisp/*.el.
- Do we need `site-lisp-collect-recursivly'? We traverse into
subdirectories at line 92.
--
TAKAHASHI Yoshio <yfb02119@nifty.com>