gnu-utils

Forum: Extension Development
Topic: gnu-utils
started by: Juanito

Posted by Juanito on April 21 2008,07:00

Having being caught out again by compiling with gnu-utils loaded (thanks for the feedback Jason), I'm wondering if it makes sense to have a gnu-utils.uci that can be unloaded.

A couple of questions:

1. Was gnu-utils originally intended to be only coreutils from here < http://directory.fsf.org/project/coreutils/ > ?
2. Given that there are probably more extensions containing xlibs, gtk2, etc than when gnu-utils was created, does it make sense to have only coreutils in gnu-utils?
3. Without having looked yet, I presume gnu-utils.dsl overwrites some apps in the base dsl - in the uci, I guess it would be OK to add a load script that moves these apps to /tmp and an unload script that moves them back?

The reasoning behind all this is that I need gnu-utils loaded at boot to load alsa at boot and (I guess) I also use some of the fully fledged (versus busybox) utilities without realising it, but I also need a way to unload gnu-utils to check that extensions do not depend on it.

Posted by roberts on April 21 2008,07:17

The original concept of gnu-utils were to be the gnu versions of that which busybox replaced - period.

While the extension managment was not under my control, gnu-utils became a catch-all of stuff, including X.

I would think it would be worthy to clean up gnu-utils to be that which it was meant to be. This would reduce conflicts but likely break old(er) extensions.

If gnu-utils is cleaned up, then we would need some effort to test and correct and/or remove old extensions that depended on a catch-all gnu-utils.

Posted by mikshaw on April 21 2008,12:55

coreutils is already a mountable extension...I just got the latest release yesterday so there might be an update to the extension if people are actually using it. Might also look into updating some of the other basics like findutils and procps.

As robert said gnu-utils is much more than coreutils. But it is *less* than a full busybox replacement, as some busybox applets remain unchanged after installation.

I also agree that gnu-utils could be improved, among other things by making a uci version. Another suggestion I would have is to separate a few of the apps into individual packages. I extracted wget from gnu-utils and use that almost daily (busybox wget is kinda lame in my opinion), but hardly ever use gnu-utils itself.

Posted by Juanito on April 21 2008,13:26

Quote

- now that sounds like a good idea. Now I know that coreutils.uci exists I'll try it out instead of gnu-utils, though I guess I'll still need find.

I guess I also need to check if the alsa boot code looks for gnu-utils or just the app(s) it requires from gnu-utils in which case coreutils.uci might do the job.

Posted by mikshaw on April 21 2008,19:23

Quote

I also need to check if the alsa boot code looks for gnu-utils

yes:

Code Sample

grep -qw gnu-utils /tmp/mydsl.map || [ -f "$MYDSLINSTALLED"/gnu-utils

But you could make up your own boot code and check for it from bootlocal.sh, using the alsa code from /etc/init.d/dsl-config (lines 449-457 in DSL 4.2)

Posted by Juanito on April 23 2008,14:39

I played around with coreutils.uci - as said, it could use findutils, sed, wget, etc in order to be a gnu-utils (less xlibs) replacement.

To avoid path errors, it would be good if there were a loader script to move the busybox symlinks to /tmp and then move them back again on unloading (assuming the coreutils apps were linked to /opt/bin)...

Thanks for the pointers on alsa - I didn't try yet, but I guess using alsa-init once coreutils.uci is loaded would probably do the trick.

Posted by mikshaw on April 23 2008,17:10

I just compiled the latest wget and procps yesterday, using compile-3.3.5.uci + term.h and the ncurses headers from gcc1-with-libs. They'll be submitted as uci extensions soon, but I'm about to do coreutils and findutils and hopefully upload them all together.

Quote

To avoid path errors, it would be good if there were a loader script to move the busybox symlinks to /tmp and then move them back again on unloading (assuming the coreutils apps were linked to /opt/bin)

The existing coreutils.uci was made long before /opt/bin was added to PATH. Even after it was added, though, I'd thought about this and decided to stick with the current arrangement for future releases. The /opt/bin directory is at the end of PATH, as you are apparently aware, so those files would not be used without removing the original symlinks (as you said). However, I don't personally like this idea for two reasons: it's simpler to just modify the PATH variable (The process is stated in the info file), and the directories containing the busybox symlinks may be read-only (as they almost always are on my system).

Posted by Juanito on April 24 2008,03:49

Quote

...using compile-3.3.5.uci + term.h and the ncurses headers from gcc1-with-libs.

- is there are problem with the ncurses headers in compile-3.3.5?

Posted by mikshaw on April 24 2008,11:58

To be honest, I simply assumed it wasn't there because this was the second application I'd tried to build that said it couldn't find ncurses.

Now that I think of it, with procps I eventually had to look into the Makefile and discovered that ncurses was set using an absolute path for some silly reason. At that point I should have thought to double-check if the headers were in compile-3.3.5.

I can't remember right now what the first curses app was...may have been htop, since I know that's one I've recently failed to build. I just know that twice in the last week or so I've had a compile failure due to a missing ncurses.h
I'll try to find another to test sometime soon.

Keep in mind that I'm no wiz when it comes to understanding how the whole compiling thing works. All I do is export CFLAGS, CPPFLAGS, and LDFLAGS and then try the build.

Posted by Juanito on April 24 2008,13:10

I've had a few compile errors due to issues over ncurses - doing the following sometimes fixes it:

Code Sample

export LIBS=" -lncurses"

To see if there was a problem using compile-3.3.5 with procps, I tried with procps-3.2.7 and got the following error:

Code Sample

$ make SHARED=0
...
pwdx.c: In function `main':
pwdx.c:38: error: `PATH_MAX' undeclared (first use in this function)

- fixed by adding the following to pwdx.c:

Quote

#include <sys/param.h>

- but I didn't get any errors involving ncurses (without the export LIBS= mentioned above)?

Note (I think) the ncurses headers in gcc1-with-libs are for a later version of ncurses than that in dsl (libncurses.so.5.3), but that probably doesn't change anything...

Posted by mikshaw on April 24 2008,13:59

I got it to find curses by adding -I/opt/compile-3.3.5/include/ncurses to the compiler flags. Unfortunately there seems to be other troubles with htop-0.7, causing another thing to break when I fixed one thing, and the first thing would break again when I fix the second thing. I had no trouble building htop-0.6.6, so I don't think it's the build environment.

I fixed the pwdx error (also procps-3.2.7) by changing the PATH_MAX variable to 4096, as it was listed in param.h
I never thought about simply including the header, which would have been the sensible solution =o)

Also term.h was needed for top, and I didn't find it in compile-3.3.5

Did you make a new version of the compile package? I wonder if that might account for our differences. I'm using one dated 2008/02/10 (seems to be the first release).

Posted by Juanito on April 24 2008,14:10

Quote

Also term.h was needed for top, and I didn't find it in compile-3.3.5

- thanks for the feedback, I need to look at this, it should have come from compiling ncurses-5.3 and thus be in compile-3.3.5

Edit: Just checked - term.h is in /opt/compile-3.3.5/include/ncurses??

Quote

Did you make a new version of the compile package? I wonder if that might account for our differences. I'm using one dated 2008/02/10 (seems to be the first release).

- the latest version was posted about a week ago. This being said, the updates were to the apps used to compile, not to any libs/headers.

Posted by ^thehatsrule^ on April 24 2008,17:38

fyi, iirc PATH_MAX errors are due to the "include bug"

Posted by Juanito on April 24 2008,17:43

Agreed, but not fixed by

Code Sample

$ cp -a /opt/compile-3.3.5/include /opt
$ export CPPFLAGS=-I/opt/include

... for some reason

Posted by mikshaw on April 24 2008,21:32

Quote

Edit: Just checked - term.h is in /opt/compile-3.3.5/include/ncurses

That would explain why it didnt work for me at the time, since I merely copied the [n]curses*.h files from gcc1-with-libs without realizing that 1) curses was already in compile-3.3.5, and 2) term.h was part of ncurses

Posted by mikshaw on April 24 2008,23:08

You had me scratching my head for a minute just now. While messing around with the wget extension I thought I'd check its info file (not the mydsl info, but the texinfo) to see how easily it would be to convert it to a format that could be easily read in DSL (the wget info provides a lot more information than the man page). To my surprise it displayed natively, or so it seemed. After a minute I thought to check it and it turned out that info is included with compile-3.3.5

I'm wondering if this is necessary. Info is definitely a good app to have, but does it have anything to do with compiling? It might be better suited as a separate extension.

Sorry I've been putting a lot of focus on compile here, when the topic should be gnu-utils.

Posted by Juanito on April 25 2008,04:03

Quote

I'm wondering if this is necessary. Info is definitely a good app to have, but does it have anything to do with compiling? It might be better suited as a separate extension.

- I seem to remember Curaga also mentioned this. I believe this comes from texinfo. It was added because a few apps I've compiled required texinfo before they would compile without errors. Perhaps the need could be avoided by the appropriate use of --disable-docs type switches?

Quote

Sorry I've been putting a lot of focus on compile here, when the topic should be gnu-utils.

- No problem, I already learnt several things from this thread.

Posted by curaga on April 25 2008,13:19

They actually often won't even check for texinfo, they just assume it exists.
My way around it, when I don't need the info files, is creating this as "makeinfo":

Quote

#!/bin/sh
exit 0

This satisfies the rude packages, as well as the well mannered ones that actually check for texinfo.
Some info pages are still installed, but they can be removed easily.

Posted by mikshaw on April 25 2008,13:29

On a semi-off-topic note, I found that info files are easily read in a text editor. It seems that the formatting done to them is little more than splitting the file into pages.

While I dislike the navigation (or apparent lack of it) in Info, they often provide much more information than man pages do, so when info pages are created I tend to keep them on hand.

Posted by Juanito on May 04 2008,15:21

I've been experimenting with compiling without gnu-utils loaded - several apps require the full version of sed to compile and some apps require something (the error message does not say what) in gnu-utils to compile.

So, the morale of the story seems to be that coreutils - plus findutils and a couple of other things would be a useful tool.

Posted by roberts on May 05 2008,18:39

Another BIG issue with gnu-utils...

Looks like a snapshot of /usr/X11R6/lib/X11 is included in gnu-utils.
Why? I don't have a clue. That has nothing to do with busybox. Likely a cude way to add some additional fonts?

Anyway, It has caused me much grief and I am cutting it from gnu-utils.

If it beaks some extensions, then post here and let me know.

BTW, a question for you font experts? Is there an easy way to see the fonts that a binary is going to be needing?

Perhaps a better method is to build an extension for adding fonts?

Posted by chaostic on May 08 2008,23:48

Quote (roberts @ May 05 2008,14:39)

Anyway, It has caused me much grief and I am cutting it from gnu-utils.

If it beaks some extensions, then post here and let me know.

Will do. (Actually, I was a bit surprised about the new size (1.2 vs 9) that I am used to seeing, when I downloaded it).

Anyway, quick question, is it necessary to run mkwritable when loading gnu-utils.dsl to a toram session?

Posted by mikshaw on May 09 2008,02:08

Quote

is it necessary to run mkwritable when loading gnu-utils.dsl to a toram session?

Yes. All toram does is load the KNOPPIX file into ram; the directories that are read-only are still read-only. Most of the directories are merely links to /KNOPPIX/* initially, regardless of whether you're running directly from CD or toram. The mkwriteable script replaces these links with writable directories, and populates those directories with links to the individual files in /KNOPPIX/* directories.

If you wish to avoid mkwriteable, you must use extensions that do not write outside of /opt: uci, unc, or tar.gz

Posted by chaostic on May 17 2008,01:25

Quote (mikshaw @ May 08 2008,22:08)

Quote

is it necessary to run mkwritable when loading gnu-utils.dsl to a toram session?

I ask because the update to gnu-utils script does it (calls mkwriteable, then mydsl-install), while reading through the mydsl-load it does not. (I save a copy of gnu-utils.dsl on my flash instead of redownloading it each time)

Edit: Actually it does. mydsl-load -> mydsl-install which calls mkwriteable.

So then a better question, why call mkwriteable twice? (Once in gnu-utils-restore, then again in mydsl-install)

Posted by roberts on May 17 2008,02:30

Any .dsl calls that script whether from boot or on-demand mydsl loading of a .dsl. However, it is only actually performed only once. Look at said script, if previously run, it immediately exits.