[Freedos-user] command.com (Freecom) main environment

[Bertho Grandpied]

On Fri, 5 Jul 2013 15:45:34 GMT
"Bret Johnson" <***@juno.com>
Hi Bret ! (long time, no see...)

... a user who is able, one way or another, to have usable RAM
mapped above the 640 k so-called limit into the "video memory'
segments, up to 736 k (B7FFF), will be forced to use the added
memory as "UMB"s instead of an extension of *contiguous* so-called
conventional mem.

Is this what you're trying to do (get more than 640k without
using UMB's), and why you don't like the XBDA / master
environment being at the top of conventional memory?

I was giving this as one example of why FreeDOS having
relegated the master environment copy to the top of conventional
mem /can/ break things. I could have given quite different examples,
nor exaples I could name old programs still available which were written
as early as the times of MS DOS 2 (!) with the implied assumption that
Command's env block follows immediately above its PSP's block.

The exact reason which made me stumble on this non-conformity of FreeCOM's does not need to be discussed in detail, it is complex and
would derail the discussion. Fact is, some project someone else is building has elected to use a FreeDOS boot disk for a basis, and this
bizarre, unmotivated, design of FreeCOM definitely interferes
negatively with some aspects of the project. It isn't ruining all,
and we could well use another shell than FreeCOM - in fact, I think
the project will be its own shell for a self-contained diskette or
USB-stick of sorts.

Now see! you've driven me into digressing from my initial purpose here again ;=)

Back to FreeCOM : of course we, app programmers, could devise a utility
for the purpose of moving that darned environment back to where it
belongs, i.e., low contiguous DOS-managed memory.
Leaving apart the fact that doing so from a transient DOS program and
without leaving holes is not absolutely a trivial programming task, we
would be still faced with the problem that, how are we supposed to
reliably /find/ and adjust the pointer(s) that of necessity FreeCOM internally keeps to the environment !

Clearly IMHO it is Freecom's responsibility (i.e., of whoever is tasked with
debugging, maintaining and enhancing FreeCOM) to provide the solution,
be it a command line option, a permanent fix, or even an API.

Hoping this could be at least some food for thought, and

Taking the leave for now

[Eric Auer]

Hi Bertho,

this thread seems to mix two topics: MS DOS command.com keeps
transient data in non-MCB areas. When, after running commands,
that area is found damaged, contents are dropped and reloaded
or recreated. As Tom already pointed out, the location of the
master env in FreeCOM is "near the end of the free DOS RAM",
but only AFTER you get the chance to load e.g. UMB drivers.
Also, the FreeDOS kernel allows you to control whether the
BIOS XBDA data should be relocated. So FreeCOM might differ
from MS DOS, but still behave in a good way.

...this bizarre, unmotivated, design of FreeCOM definitely interferes
negatively with some aspects of the project. It isn't ruining all,

Bizarre yes, unmotivated no. For example XMS swapping is one
very useful feature of the complex FreeCOM memory handling.

and we could well use another shell than FreeCOM - in fact, I think
the project will be its own shell for a self-contained diskette or
USB-stick of sorts.

Sounds like a nice project.

Back to FreeCOM : of course we, app programmers, could devise a utility
for the purpose of moving that darned environment back to where it
belongs, i.e., low contiguous DOS-managed memory.

It is still obscure to me why that "darned" environment is so
much in the way for your project where it is with FreeCOM...?

Leaving apart the fact that doing so from a transient DOS program and
without leaving holes is not absolutely a trivial programming task, we
would be still faced with the problem that, how are we supposed to
reliably /find/ and adjust the pointer(s) that... FreeCOM internally keeps

I may be mistaken, but should not just the PSP point to the environment?

Clearly IMHO it is Freecom's responsibility (i.e., of whoever is tasked with
debugging, maintaining and enhancing FreeCOM) to provide the solution,
be it a command line option, a permanent fix, or even an API.

See above. Also, nobody is "tasked" with FreeCOM, as this is
not a commercial product. Unfortunately, very few people who
are FreeCOM experts are around here recently... On the other
hand, maintenance is so slow because there was so little left
to improve. As mentioned earlier in this thread, there could
be some regressions between 0.82, 0.83 and 0.84, so you might
find an older version to be more of your taste. Not regarding
the master environment, but in general.

Regards, Eric

PS: I totally agree that the tab completion beep should be
disable-able and/or switchable to DOS console or BIOS BEL
char output, as it can be annoying and even cause hangs.

[Matej Horvat]

Post by Eric Auer
As mentioned earlier in this thread, there could
be some regressions between 0.82, 0.83 and 0.84, so you might
find an older version to be more of your taste. Not regarding
the master environment, but in general.
PS: I totally agree that the tab completion beep should be
disable-able and/or switchable to DOS console or BIOS BEL
char output, as it can be annoying and even cause hangs.

Wrong thread/list?

[Bertho Grandpied]

Hi Eric,
Yep, things got mixed, I suppose it's my fault.
Consequently I'll try to leavo out - not quote - the bits
which, though they may be interesting, I think irrelevant herein

(snip...)

Post by Eric Auer
As Tom already pointed out, the location of the
master env in FreeCOM is "near the end of the free DOS RAM",
but only AFTER you get the chance to load e.g. UMB drivers.

And as I must stress once again, I am considering the /alternate/
case, when no UMBs will be provided for DOS use. I have been suggesting that in this case, FreeCOM ought to allocate the block that it reserves for
storing the main (1st level) environment area. Similar to what other
reference DOSes have done forever (well, starting since MS-DOS 2, I think
DOS 1 did not have the concept of "environment variables" later "borrowed" from Unix...

Post by Eric Auer
Also, the FreeDOS kernel allows you to control whether the
BIOS XBDA data should be relocated.

This was a lateral question, but still : how ? the kernel we use doesn't seem to obey a 'Switches=/E' directive in Config.sys, that is the MS-DOS way.

Post by Eric Auer
So FreeCOM might differ from MS DOS, but still behave in a good way.

I beg to differ vigorously - it behaves in a bad (tm) way.

Post by Eric Auer

...this bizarre, unmotivated, design of FreeCOM definitely interferes
negatively with some aspects of the project. It isn't ruining all,

Bizarre yes, unmotivated no. For example XMS swapping is one
very useful feature of the complex FreeCOM memory handling.

Like 4DOS by the way. I fail to see what XMS swapping (nice to have) is to do
with my critique, though...

Post by Eric Auer

Back to FreeCOM : of course we, app programmers, could devise a utility
for the purpose of moving  that darned environment back to where it 
belongs, i.e., low contiguous DOS-managed memory.

It is still obscure to me why that "darned" environment is so
much in the way for your project where it is with FreeCOM...?

Suffice it to say it is an impediment to that project, plus a non conformance that will break other DOS apps for no real reason (that I can perceive)

Post by Eric Auer

Leaving apart the fact that doing so from a transient DOS program and
without leaving holes is not absolutely a trivial programming task, we
would be still faced with the problem that, how are we supposed to 
reliably /find/ and adjust the pointer(s) that... FreeCOM internally keeps

I may be mistaken, but should not just the PSP point to the environment?

You are mistaken indeed. Command's, or FreeCOM's own PSP would point to /its/ initial environment (if there was any established by Config.sys command, as has been possible in /MS/DOS since DOS 6, I know not whether FreeDOS has a similar concept. Let's not digress again!). Thus, the environment pointer inside the command processor's (shell's) PSP is either NUL, else pointing to an initial environment of no significance nor use.(Digressing again, I have been known to free that useless initial environement).

/The/ "master environment" is a brand new one built by Command while it
initialises things, and sized according to Command's command line
parameters (if this is the primary shell, in turn lifted from the
SHELL line in config.sys).

Command / FreeCOM has to ask a block from DOS using the usual allocation
functions 48h etc, with appropriate choices of "strategy".
To sum up, repairing FreeCOM may be as simple as it passing the proper
allocation strategy, i.e., don't include UMBs + find lowest suitable block
as part of its DOS calls reserving that block.
to the shell from DOS System initialisation.)

Post by Eric Auer

Clearly IMHO it is Freecom's responsibility (i.e., of whoever is tasked with 
debugging, maintaining and enhancing FreeCOM) to provide the solution,
be it a command line option, a permanent fix, or even an API.

See above. Also, nobody is "tasked" with FreeCOM, as this is
not a commercial product.

I meant tasked in the sense of being the official maintainers, not as in
paid for doing it.

Post by Eric Auer
Unfortunately, very few people who
are FreeCOM experts are around here recently... On the other
hand, maintenance is so slow because there was so little left

to improve.

Understood. Maybe if whoever is familiar with FreeCOM sources would
consider the elements I have tried to provide, they would not find it
unsurmountable to repair (or diplomatically, revise) this aspect.

Bye !

[Bret Johnson]

Post by Bertho Grandpied
I was giving this as one example of why FreeDOS having
relegated the master environment copy to the top of conventional
mem /can/ break things

[Bertho Grandpied]

Guys...

Casually peeking at Freecom source, branch MAIN, init.c v 1.31,

Freecom's rev 1.31 (Excerpt, from Sourceforge CVS, w/ line numbers)
[code]
437 env_resizeCtrl |= ENV_USEUMB | ENV_ALLOWMOVE | ENV_LASTFIT;
438 if(forceLow)
439 env_resizeCtrl &= ~ENV_USEUMB;
440 if(newEnvSize > 16 && newEnvSize < 32767)
441 env_setsize(0, newEnvSize);
[/code]

et caetera, etc...

I'm not an expert in C, not even close to a novice...

With this duly noted, it appears it won't be hard for you Freecom
code contributors / revewers (is this not you, Tom E.?) to edit the above
module and possibly a handful of dependencies so that FreeCOM eventually
will request the environment block using "Firstfit" instead of "Lastfit"
strategy, at least when allocating from low memory. I don't care as much
if it uses "lastfit" when the request is effectively UMBs only, although
I'm not sure what is to be gained by using 'lastfit' even in upper memory.

Hoping someone will take the challenge,

w/ respect

[Bertho Grandpied]

"Bret Johnson" wrote :

[Bret Johnson]

I see... following a very famous lead, you think that 640 kilo
bytes are more DOS memory than anybody will ever want :=)

No, that's not what I'm saying at all. One thing I am trying to say is that if you're going to use DOS memory, then you should use the memory allocation functions provided by DOS that already take things like the XBDA and master environment into account. Your advisee may may not /want/ anything (except what he wants to put there) at the top of conventional memory, but there's actually no legitimate reason other programs (including the BIOS and DOS) can't put anything there.

If I were doing this, I wouldn't expect FreeDOS to change the way it does things -- I would change how my program works. This is not a FreeDOS "bug" that needs to be fixed, IMO. I agree it would probably nice to at least have an option, but I still wouldn't classify it as a "bug".

I hardly see how a regular program launched by FreeCOM could
relocate FreeCOM's master environment in a safe and generic enough
way, that is to say, without runtime "patching" FreeCOM at
specific, version dependent address(es).

You're correct --it couldn't. I'm merely trying to offer up suggestions that may trigger some sort of alternative in your thought process (like maybe even doing something in CONFIG.SYS before FreeCOM ever even loads). E.g., install a "temporary" device driver that reserves the upper part of conventional memory so that FreeCOM can't load there. Maybe you can come up with a different/better idea.

[Bertho Grandpied]

Tom,

How many users do you have ?

I have no idea - and don't care.

Of these, how many understands this level of detail, /and/ in addition, will care ?

answering this question would imply that /you/ understand the problem.
you don't.

Now, is that not arrogance ! This kind of remarks is best avoided,
it is not a way to prove your competence which has not been disputed anyway.

I just wrote the code to put the environment where it is (in 2001),
that's all. and it's placed where it is for a *very* good reason.
just think a bit about it

If there is such a reason, you should be able to state it, for the

benefit of us all who are not geniuses like you. Please *enlighten* me !

the FreeDOS kernel relocates the XBDA.

Really ? Will it ? The kernel I'm using will /not/ do it, certainly not by default.

this is either a bug, or your XBDA behaves somewhat different.

An XBDA is an XBDA is an XBDA, furthermore this is observed on more
than one machine.
I"m using a kernel v. 2040 with XMS swapping compiled in. An
" SWITCHES = /E " directive in Config.sys is ignored silently -
no error flagged. My conclusion has been that the option was reserved
but unimplemented. Is kernel 2041 different in this respect ?

an unrelated story.

Agreed.

Why these latter limits would be thus hard-coded is beyond me.

b7ff is shorter then 'the highest location in conventional memory'

Really ? 80x86 systems /other/ than the "standard Wintel PC" (a moving goal anyway) should be able to run a DOS efficiently. Think out of the matrix!
let's not be formatted by years of IBM/Microsoft/Intel/PCI-SIG...whoever's... idea of what a PC/DOS machine
must be like. Rant and digression over.

... Snipping repetitive stuff...

Casually peeking at Freecom source, branch MAIN, init.c v 1.31,

...

I'm not sure what is to be gained by using 'lastfit' even in upper memory.

...

Hoping someone will take the challenge,

left as an exercise to the reader. suffice it to say: you wouldn't like the adverse effects.

Is there a FreeCOM 'blueprint' or design document for FreeCOM
other than the code comments ?

You can't escape having to explain what "adverse effects" you were evoking, now anyway.

Bye !

[Eric Auer]

Hi Bertho,

maybe Tom does not have the patience to explain you
why there are good reasons why FreeDOS does things
the way they are done, but you can trust him :-) If
you take the time to FULLY understand the issue and
then have exact ideas on how to do things yet better,
you are most welcome to submit patches or explain an
algorithm change and why that new algorithm is good.

Cheers, Eric



PS: XBDA and moving it is not necessarily trivial.

Post by Bertho Grandpied
I"m using a kernel v. 2040 with XMS swapping compiled in.

XMS swapping is done by FreeCOM, not the kernel...

Post by Bertho Grandpied
" SWITCHES = /E " directive in Config.sys is ignored silently
case 'E': /* /E[[:]nnnn] Set the desired EBDA amount to move in bytes */
{ /* Note that if there is no EBDA, this will have no effect */
int n = 0;
if (*++pLine == ':')
pLine++; /* skip optional separator */
if (!isnum(*pLine))
{
pLine--;
break;
}
pLine = GetNumArg(pLine, &n) - 1;
/* allowed values: [48..1024] bytes, multiples of 16
* e.g. AwardBIOS: 48, AMIBIOS: 1024
* (Phoenix, MRBIOS, Unicore = ????)
*/
if (n == -1)
{
Config.ebda2move = 0xffff;
break;
}
else if (n >= 48 && n <= 1024)
{
Config.ebda2move = (n + 15) & 0xfff0;
break;
}
/* else fall through (failure) */
}
if (Config.ebda2move)
{
ebda_size = ebdasize();
ram_top += ebda_size / 1024;
if (ebda_size > Config.ebda2move)
ebda_size = Config.ebda2move;
}

...

Post by Bertho Grandpied
if (ebda_size) /* move the Extended BIOS Data Area from top of RAM here */
movebda(ebda_size, FP_SEG(KernelAlloc(ebda_size, 'I', 0)));

You also want to read some of the related code. You are
right that there is no feedback telling you step by step
what happens with the EBDA, but making the kernel that
verbose would also make it unnecessarily large.

Post by Bertho Grandpied

Why these latter limits would be thus hard-coded is beyond me.

b7ff is shorter then 'the highest location in conventional memory'
Really ? 80x86 systems /other/ than the "standard Wintel PC" (a moving goal

Well... Anyway. The DOS running on not so standard PC,
say TI PRO or TANDY, also was not vanilla but had to
hardcode the non-standard aspects of those platforms.

Post by Bertho Grandpied
Is there a FreeCOM 'blueprint' or design document for FreeCOM
other than the code comments ?

There are some text files, wiki / homepage documents...
But generally speaking: If it is not part of the source
download, it is probably not included.

[Tom Ehlert]

Bertho ???,

Post by Bertho Grandpied

Post by Bertho Grandpied
Casually peeking at Freecom source, branch MAIN, init.c v 1.31,

...

Post by Bertho Grandpied
I'm not sure what is to be gained by using 'lastfit' even in upper memory.

...

Post by Bertho Grandpied
Hoping someone will take the challenge,

left as an exercise to the reader. suffice it to say: you wouldn't like the adverse effects.

Is there a FreeCOM 'blueprint' or design document for FreeCOM
other than the code comments ?

nothing except 'look at how MS command behaves'

Post by Bertho Grandpied
You can't escape having to explain what "adverse effects" you were evoking, now anyway.

command.com is a 'normal' program. just allocating DOS memory will
give you an environment at ~1800:0. not such a good idea.
usually ~9f00:0 is a much better place.
and juggling memory around (beyond what is already done) was so far
never necessary (and still isn't)

Tom

[Bertho Grandpied]

Hi, Eric Auer !

Post by Eric Auer
PS: XBDA and moving it is not necessarily trivial.

Our EBDA mover is working flawlessly AFAICT.Old code of mine actually,
written for MSDOS 5+ before MS introduced the switches=/E option (or
before I was aware of the switch?). Last week when I had discovered
that the respective FreeDOS kernel's "switches" doesn't work (see below),
we merged in new code to deal with it.

My and the project's intention as a way to somehow repay in part what
FreeDOS is giving us was to offer this generic MSDOS+FreeDOS EBDA mover
for inclusion at the FD repositories as soon as all the project's people
has had a chance to field tested the release candidate on as many DOS
machines and configurations as they can.

I must say, though, the reception which I got from Herr Ehlert on this list is making me wonder whether spontaneous contributions made in
good faith are welcome and / or opportune.

Post by Eric Auer
maybe Tom does not have the patience to explain you
why there are good reasons why FreeDOS does things
the way they are done, but you can trust him :-)

As in, I should /trust/ someone blindly who /starts/ interacting by affirming
that I /do not understand/ the point ? Which is funny because, as much
as I claim I /do not/ grasp the C language, I will claim having a /good/
understanding of DOS's memory management, the API and the 'innards' as
well, and even the implementation quirks (Microsoft's). So I /might/ take
offence of what you call, mildly, Tom's lack of patience.

Further to stating that I Czerno "do not understand", and that he Tom
"does not care" about your users, I have yet to read Herr Ehlert's
statement of why it would be (dangerous? unwelcome?) for Freecom
to allocate the master environment block in low memory using "first fit".

I'm open and ready to accept sound reasons, which must be FreeCOM specific then,
just not the "you wouldn't be happy" stance ! Am I bizarre ?

Post by Eric Auer
XMS swapping is done by FreeCOM, not the kernel...

Indeed, my typo.

Post by Eric Auer

Post by Bertho Grandpied
" SWITCHES = /E " directive in Config.sys is ignored silently

...

Post by Eric Auer
/* allowed values: [48..1024] bytes, multiples of 16
...
You also want to read some of the related code. You are
right that there is no feedback telling you step by step
what happens with the EBDA, but making the kernel that
verbose would also make it unnecessarily large.

Well, I won't enter into that code in detail - I don't do *C*, remember !

but the FreeDOS kernel mover /does not work/ on *real* machines, ours /does/ work ;=)

At first sight, FreeDOS makes assumptions which are much
too restrictive (in present days, XBDAs are over 1 k byte in length,
6 kilobytes aren't rare.)

Post by Eric Auer

Post by Bertho Grandpied
Is there a FreeCOM 'blueprint' or design document for FreeCOM
other than the code comments ?

There are some text files, wiki / homepage documents...
But generally speaking: If it is not part of the source
download, it is probably not included.

This is a problem with many projects of course.

Well... the project which I keep mentionning won't include a "COMMAND"
processor in the final distribution as self sufficient bootable images for a floppy or USB key. The final user may want to add one and we certainly
want a command shell during project building. I'll be recommending 4DOS
for internal use - license allowing.

In case a future version of FreeCOM finally can be persuaded to locate
the ENV low, like MS and all other Command.com flavours rightly do,
shall we reconsider.

Kind regards

[Rugxulo]

Hi,

Post by Bertho Grandpied
I must say, though, the reception which I got from Herr Ehlert on this list is making me wonder whether spontaneous contributions made in
good faith are welcome and / or opportune.

Patience, young padawan. Things like this take time and thought (and
research and testing).

If you draw up a patch and it isn't accepted upstream into SVN, I can
still mirror it somewhere (e.g. iBiblio) for you.

Or you can do almost anything with it anyways, it's free/libre. There
can be no one stopping you from contributing (somewhere, somehow).

Post by Bertho Grandpied

Post by Eric Auer
maybe Tom does not have the patience to explain you
why there are good reasons why FreeDOS does things
the way they are done, but you can trust him :-)

As in, I should /trust/ someone blindly who /starts/ interacting by affirming
that I /do not understand/ the point ?

Give him the benefit of the doubt, as he is one of the resident
experts around here who has contributed a lot. But even the smartest
person in the world gets tired, too busy with "real life", or just
forgets some minor details from years ago.

Post by Bertho Grandpied
Further to stating that I Czerno "do not understand", and that he Tom
"does not care" about your users, I have yet to read Herr Ehlert's
statement of why it would be (dangerous? unwelcome?) for Freecom
to allocate the master environment block in low memory using "first fit".
I'm open and ready to accept sound reasons, which must be FreeCOM specific then,
just not the "you wouldn't be happy" stance ! Am I bizarre ?

Yes, of course, technical explanations (or how to test for
correctness) would be ideal, but he may not have time for that nor
remember the details.

Post by Bertho Grandpied
Well... the project which I keep mentionning won't include a "COMMAND"
processor in the final distribution as self sufficient bootable images for a floppy or USB key. The final user may want to add one and we certainly
want a command shell during project building. I'll be recommending 4DOS
for internal use - license allowing.

4DOS is ambiguously licensed. I don't really recommend it, though
there aren't a lot of "full" shell replacements available. If you can
avoid some (most?) .BAT internal stuff, you may find it easier to
replace:

http://www.ibiblio.org/pub/micro/pc-stuff/freedos/files/dos/command/contrib/

Post by Bertho Grandpied
In case a future version of FreeCOM finally can be persuaded to locate
the ENV low, like MS and all other Command.com flavours rightly do,
shall we reconsider.

Again, I take this to mean that (admittedly) FreeCOM is too hard to
rebuild (preferably with TurboC). If you need help with this, feel
free to try and ask specifically for assistance. (It's definitely what
I would call slightly annoying, but it's definitely not impossible to
rebuild either.)

[Bertho Grandpied]

Post by Tom Ehlert
Bertho ???,

You may call me Czerno, Herr Ehlert

Post by Tom Ehlert

Post by Bertho Grandpied
You can't escape having to explain what "adverse effects" you were evoking, now anyway.

command.com is a 'normal' program. just allocating DOS memory will
give you an environment at ~1800:0. not such a good idea.

You are joking, Herr Ehlert, richtig ?
Launching a basic FreeDOS+FreeCOM virtual machine while I'm typing...
No upper memory. Using (Jack Ellis's, I think ) XMGR.SYS.
MEM /D indicates the ENV would be at *526:0*, /not/ such a /bad idea/.
And this is /with/ VMware's BIOS 5 kilobytes EBDA relocated low, mind you.

Of course the kernel is in HMA, which may be what your reply eluded !

And EVEN if for some reason HMA was not available or not given to the DOS kernel, what makes you deem an environment at ~1800:0 not such a good idea ?

I this all your deep reason for forcing the master ENV up at 9xxxx ?
In which way other than your respectable personal preference is it better?

This is highly ridiculous. At least provide a choice. Leave it to
power /users/ to determine for themselves what memory layout is
best in /their/ situation.

Ah, but - sorry, I was forgetting - you /don't care/ much about your users.

No wonder you don't have many.

Post by Tom Ehlert
usually ~9f00:0 is a much better place.
and juggling memory around (beyond what is already done) was so far

never necessary  (and still isn't)

How do you say "arrogance" in German, Herr /Doktor/ Ehlert ?

Wiedersehen

[Tom Ehlert]

Post by Bertho Grandpied

Post by Tom Ehlert
Bertho ???,

You may call me Czerno, Herr Ehlert

your email signature reads "Bertho Grandpied <***@yahoo.fr>"
that translates to Bob Bigfoot, right ?

Post by Bertho Grandpied

Post by Tom Ehlert

Post by Bertho Grandpied
You can't escape having to explain what "adverse effects" you were evoking, now anyway.

command.com is a 'normal' program. just allocating DOS memory will
give you an environment at ~1800:0. not such a good idea.

You are joking, Herr Ehlert, richtig ?
Launching a basic FreeDOS+FreeCOM virtual machine while I'm typing...
No upper memory. Using (Jack Ellis's, I think ) XMGR.SYS.
MEM /D indicates the ENV would be at *526:0*, /not/ such a /bad idea/.
And this is /with/ VMware's BIOS 5 kilobytes EBDA relocated low, mind you.

I recommend to apply your 'allocate low' change to FreeCOM yourself.
you will see what happens.

Post by Bertho Grandpied
Of course the kernel is in HMA, which may be what your reply eluded !

no.

Post by Bertho Grandpied
And EVEN if for some reason HMA was not available or not given to
the DOS kernel, what makes you deem an environment at ~1800:0 not such a good idea ?

you would end up with
3 K COMMAND.COM, (resident part)
100 K FREE (remainders of freecom before resizing)
1 K command.com environment (at ~1800:0)

Post by Bertho Grandpied
I this all your deep reason for forcing the master ENV up at 9xxxx ?
In which way other than your respectable personal preference is it better?
This is highly ridiculous. At least provide a choice. Leave it to
power /users/ to determine for themselves what memory layout is
best in /their/ situation.
Ah, but - sorry, I was forgetting - you /don't care/ much about your users.

your are making things up - again.

you asked

Post by Bertho Grandpied
How many users do you have ?

I answered
I have no idea - and don't care.

Post by Bertho Grandpied
How do you say "arrogance" in German, Herr /Doktor/ Ehlert ?

Arroganz.

Tom

[Bertho Grandpied]

Post by Tom Ehlert

Post by Bertho Grandpied
You may call me Czerno, Herr Ehlert

that translates to Bob Bigfoot, right ?

Ask Yahoo!... Then if you insist on calling me Bertho, be my guest.

Post by Tom Ehlert

Post by Bertho Grandpied
And EVEN if for some reason HMA was not available or not given to
the DOS kernel, what makes you deem an environment at ~1800:0  not
such a good idea ?

you would end up with
    3   K COMMAND.COM, (resident part)
    100 K FREE         (remainders of freecom before resizing)
    1   K   command.com environment (at ~1800:0)

How lame ! Of course, your Freecom shall have to play a minimum game of
releasing its own initialisation code and data, resizing and possibly moving things along. Basic DOS system coding skills, not rocket science.

This is/the/ official shell we are talking of, not some half baked
throwable transient proggy, it deserves having a minimum of /intelligent
design/ applied :=)

There are even other enhancements that could be made in the same domain,
an easy one is to give back the /initial/ environment (if any) received by
your Command from its caller (or from Sysinit). It is of no use after the
Master ENV is built. Freeing it would give FreeDOS an edge over MS
Command. But the important is the above. You (or was it Eric) repeated
that FreeCOM had to follow MS-Command, but in this respect it doesn't
even start to try, I am /sorry/ to observe.

What is the legal status of 4DOS in relation to FreeDOS ? There's a fully baked
product, could it become /the/ main FD shell ?

Post by Tom Ehlert

Post by Bertho Grandpied
How do you say "arrogance" in German, Herr /Doktor/ Ehlert ?

Arroganz.

Itself ! I may sound harsh, but being accused of ignorance by
more ignorant is the only word of excuse I will utter.

Let's forget /name calling/ ? If you agree, I'm your man

[Rugxulo]

Hi,

Post by Bertho Grandpied
What is the legal status of 4DOS in relation to FreeDOS ? There's a fully baked
product, could it become /the/ main FD shell ?

Unlikely to become the main shell (though that's not my decision
anyways). I'm honestly not sure how free/libre it is. IIRC, the
original license (when sources were released) was quite contradictory,
so I'm not sure if "commercial use" is allowed, which is indeed a big
restriction that OSI and FSF rally against. (Not to mention requiring
non-free tools. I don't think the partial patch to use OpenWatcom was
ever very successful, but I never tried, and certainly Lucho only used
old official MS tools.)

I don't know the details, I'm no lawyer. It's unlikely to ever change.
Feel free to contact the copyright holder (Rex Conn? JPsoft?) for
clarification, but then again, they may not respond (in any useful
manner), so don't get your hopes up.

http://jpsoft.com/company/contact-jp-software.html

Sorry to be so pessimistic.

[Tom Ehlert]

Post by Bertho Grandpied

Post by Tom Ehlert
you would end up with
    3   K COMMAND.COM, (resident part)
    100 K FREE         (remainders of freecom before resizing)
    1   K   command.com environment (at ~1800:0)

How lame ! Of course, your Freecom shall have to play a minimum game of
releasing its own initialisation code and data, resizing and
possibly moving things along. Basic DOS system coding skills, not rocket science.
This is/the/ official shell we are talking of, not some half baked
throwable transient proggy, it deserves having a minimum of /intelligent
design/ applied :=)

it looks like a real experienced programmer (you) has to show us
confused youngsters how it should be done!

Post by Bertho Grandpied
There are even other enhancements that could be made in the same domain,
an easy one is to give back the /initial/ environment (if any) received by
your Command from its caller (or from Sysinit). It is of no use after the
Master ENV is built. Freeing it would give FreeDOS an edge over MS
Command. But the important is the above.

I'm deep impressed. we have a black belt grandmaster on visit. unfortunately too
busy to work, but we are so happy to receive good advice ;)

Post by Bertho Grandpied
You (or was it Eric) repeated
that FreeCOM had to follow MS-Command,

it was dennis

Post by Bertho Grandpied
but in this respect it doesn't
even start to try, I am /sorry/ to observe.

Post by Tom Ehlert

Post by Bertho Grandpied
How do you say "arrogance" in German, Herr /Doktor/ Ehlert ?

Arroganz.

Itself ! I may sound harsh, but being accused of ignorance by
more ignorant is the only word of excuse I will utter.

this 'more' makes me think that you should prove your competence first

Post by Bertho Grandpied
Let's forget /name calling/ ? If you agree, I'm your man

Tom

[Bertho Grandpied]

Att: Rugxulo

Hi,

Post by Rugxulo
Patience, young padawan.

I had to look that padawan up on wOOkipedia (not a typo!) :=)
<http://starwars.wikia.com/wiki/Padawan>

Post by Rugxulo
Things like this take time and thought (and

research and testing).

Accepted. I've passed the message, now letting things ripen (and tone down) as appropriate.

Post by Rugxulo
If you draw up a patch and it isn't accepted upstream into SVN, I can

still mirror it somewhere (e.g. iBiblio) for you.

I'm afraid no patch to FreeCOM is likely will be coming from me, for reasons
already stated and rehashed.

Should OTOH you (and the FreeDOS project at large) wish to offer the
free XBDA mover as a supplement/alternative to FreeDOS's internal, I'll
contact you for arranging the mirroring. It's a simple, robust and
tiny DOS device driver coded in ASM, a few hundred bytes altogether.

Post by Rugxulo

Post by Eric Auer
maybe Tom does not have the patience to explain you
why there are good reasons why FreeDOS does things
the way they are done, but you can trust him :-)

(...)

Post by Rugxulo

Post by Eric Auer
Give him the benefit of the doubt, as he is one of the resident

experts around here who has contributed a lot.

I think I did give the, whatever, benefit - but maybe it was not apparent ?

Post by Rugxulo
But even the smartest
person in the world gets tired, too busy with "real life", or just
forgets some minor details from years ago.

Sure. but the smartest person in the world won't convince me s/he
cannot discard 100 K of temporary code/data and do some memory resizing,
move part of itself out of the way if need be - before allocating memory.

It's not exactly a beginning programmer's task, nor is it rocket science either.

[...]

Post by Rugxulo

Post by Eric Auer
want a command shell during project building. I'll be recommending 4DOS
for internal use - license allowing.

4DOS is ambiguously licensed. I don't really recommend it, though
there aren't a lot of "full" shell replacements available. If you can
avoid some (most?) .BAT internal stuff, you may find it easier to

I /love/ 4DOS - been using it for 20 years - used to be NDOS.
For internal use, it must be OK, right ?
The question wrt to licensing was rather whether 4DOS.COM could be legally envisaged to become FreeCOM as FreeDOS official, or "alternate
official?" shell.

Post by Rugxulo
http://www.ibiblio.org/pub/micro/pc-stuff/freedos/files/dos/command/contrib/
Again, I take this to mean that (admittedly) FreeCOM is too hard to
rebuild (preferably with TurboC). If you need help with this, feel
free to try and ask specifically for assistance.

Having never written a line in C makes it a no-no for myself, and no programmer on their small team will take an interest in what is outside
the realm of the project.

Post by Rugxulo
(It's definitely what
I would call slightly annoying, but it's definitely not impossible to
rebuild either.)

Rebuilding is one thing, patching and debugging properly is another.
This task takes a motivated, seasoned C programmer, who preferably were
familiar already with FreeCOM. All I can do is try to argue it, knowing
well it's a possibility that no one will be convinced and undertake it.

On the other hand, if FreeCOM hasn't been revised since 2001, maybe it's
not to early for 'someone' to give the old code a look and some fresh
thought.

Regards

[Tom Ehlert]

Hi,

Post by Bertho Grandpied
Should OTOH you (and the FreeDOS project at large) wish to offer the
free XBDA mover as a supplement/alternative to FreeDOS's internal, I'll
contact you for arranging the mirroring. It's a simple, robust and
tiny DOS device driver coded in ASM, a few hundred bytes altogether.

it probably should be merged into the kernel, or actually used as a
blue print to fix the bug (?) in the current kernel that prevents the
XBDA move.

Post by Bertho Grandpied
On the other hand, if FreeCOM hasn't been revised since 2001,

nobody said that.

Tom

[Rugxulo]

Hi,

Post by Bertho Grandpied
Should OTOH you (and the FreeDOS project at large) wish to offer the
free XBDA mover as a supplement/alternative to FreeDOS's internal, I'll
contact you for arranging the mirroring. It's a simple, robust and
tiny DOS device driver coded in ASM, a few hundred bytes altogether.

A nice readme.txt telling how to use it would be most helpful. :-)
But yes, of course, anything free/libre (four freedoms) can be
mirrored.

Post by Bertho Grandpied

Post by Rugxulo
4DOS is ambiguously licensed. I don't really recommend it

I /love/ 4DOS - been using it for 20 years - used to be NDOS.
For internal use, it must be OK, right ?

I don't know, I'm no lawyer. I don't even want to think about it. It's
out of my hands anyways. (And now I remember that Bernd put it in FD
1.1 anyways, maybe as default!)

Post by Bertho Grandpied
The question wrt to licensing was rather whether 4DOS.COM could be legally envisaged to become FreeCOM as FreeDOS official, or "alternate
official?" shell.

It's not as easy as it sounds to be compatible between shells with
.BAT scripts. Personally, I stick to plain .BAT (usually FreeCOM) and
third-party tools (or external scripting languages) rather than rely
on 4DOS.

Though there's nothing technically horrible about 4DOS, but it's not
really worth using exclusively, IMO. (Though, again, it's not my
decision what FreeDOS proper does.) I do have it as an optional shell
in my CONFIG.SYS menu, but I rarely use it.

Post by Bertho Grandpied
Rebuilding is one thing, patching and debugging properly is another.
This task takes a motivated, seasoned C programmer, who preferably were
familiar already with FreeCOM. All I can do is try to argue it, knowing
well it's a possibility that no one will be convinced and undertake it.

Bart Oldeman is the dude to contact. He was the last one officially
working on it (circa late 2011). You may have to email him directly,
but again, he may be too busy these days (educated guess).

http://sourceforge.net/p/freedos/svn/1709/tree/

"freecom 2011-07-29 bartoldeman [r1696] Merged fcompl1 and
fcompl2.c to filecomp.c"

Post by Bertho Grandpied
On the other hand, if FreeCOM hasn't been revised since 2001, maybe it's
not to early for 'someone' to give the old code a look and some fresh
thought.

Not sure where you got 2001 from. It's hard to tell from (very) quick
glance, but it seems 2003 (0.82) and 2006 (0.84). I know, not much
better, but still ...! :-)

[Matej Horvat]

Hi Bertho,

I've spent the last three hours writing a tiny COM program that moves
COMMAND.COM's environment block to the lowest address possible (using
DOS's "low memory first fit" strategy). Unfortunately, it does not seem to
be able to find the environment block if used in AUTOEXEC.BAT after HIMEMX
and JEMM386 were loaded in FDCONFIG.SYS (I only tested both at the same
time, so I don't know which one causes the problem), but it works if run
after AUTOEXEC.BAT has been processed (for some reason - but by then you
may have TSRs loaded between COMMAND.COM and its environment, which may or
may not be OK for your needs) or if those two drivers are not loaded.

So for example, without HIMEMX and JEMM386, FreeCOM will allocate its
environment block at e. g. segment 9F7F, but after running the program,
the environment block will reside at segment 23B4, almost immediately
after COMMAND.COM (it depends on the order in which you load TSRs).

Please contact me off-list if you want to try it out.

Matej Horvat
http://matejhorvat.si/

[Bertho Grandpied]

Post by Matej Horvat
I've spent the last three hours writing a tiny COM program that moves
COMMAND.COM's environment block to the lowest address possible (using
DOS's "low memory first fit" strategy).

Nice !

Post by Matej Horvat
Unfortunately, it does not seem to  
be able to find the environment block if used in AUTOEXEC.BAT after
HIMEMX and JEMM386 were loaded in FDCONFIG.SYS (I only tested both at the same  
time, so I don't know which one causes the problem),

My bet would be on JEMM as the one which messes with memory control blocks.

Post by Matej Horvat
but it works if run  
after AUTOEXEC.BAT has been processed (for some reason - but by then
you may have TSRs loaded between COMMAND.COM and its environment,
which may or may not be OK for your needs) or if those two drivers are not loaded.

Yup. TSRs which are never unloaded would not be a problem, otherwise
they could leave holes, but that's life... One could even re-run your
program and try to relocate the ENV again into such a hole, if it's big
enough :=)

Post by Matej Horvat
So for example, without HIMEMX and JEMM386, FreeCOM will allocate its
environment block at e. g. segment 9F7F, but after running the program,
the environment block will reside at segment 23B4, almost immediately
after COMMAND.COM (it depends on the order in which you load TSRs).

Thanks. It would work indeed because - as I was checking a few minutes
before reading your mail - FreeCOM does maintain the master environment
pointer (its segment) in its PSP:3Ch slot, additionally FreeCOM seems
to use the contents of that 'slot', not some copy it made, whenever
it needs to access the ENV. At least a search (using debug!) did not
find the environment segment, as data, inside FreeCOM's live memory.

Of course it could be keeping the live ENV's segment in
registers, but since that is "C" it's dubious registers are used for
long time storage (contrary to what ASM code does profusely, at least mine).

Of course, I presume your program does care to adjust the PSP:3C pointer.

Post by Matej Horvat
Please contact me off-list if you want to try it out.

Well, thank you. A fully satisfactory solution ought to come from
FreeCOM's initialisation itself, though.

Regards

[Matej Horvat]

Post by Bertho Grandpied
My bet would be on JEMM as the one which messes with memory control blocks.

You're right. It is JEMM.

Post by Bertho Grandpied
Yup. TSRs which are never unloaded would not be a problem, otherwise
they could leave holes, but that's life... One could even re-run your
program and try to relocate the ENV again into such a hole, if it's big
enough :=)

Sure, you can run it any number of times.

Post by Bertho Grandpied
FreeCOM does maintain the master environment
pointer (its segment) in its PSP:3Ch slot, additionally FreeCOM seems
to use the contents of that 'slot', not some copy it made, whenever
it needs to access the ENV.

3Ch is reserved as far as I know. Do you mean 2Ch? That's what my program
modifies.

[Tom Ehlert]

Post by Bertho Grandpied
Thanks. It would work indeed because - as I was checking a few minutes
before reading your mail - FreeCOM does maintain the master environment
pointer (its segment) in its PSP:3Ch slot, additionally FreeCOM seems
to use the contents of that 'slot', not some copy it made, whenever
it needs to access the ENV.

that's how 'edit the parent/master environment' utilities work(ed),
it's 'documented undocumented' behaviour, and FreeCOM behaves and has
to behave) the same way.

Post by Bertho Grandpied
At least a search (using debug!) did not
find the environment segment, as data, inside FreeCOM's live memory.
Of course it could be keeping the live ENV's segment in
registers, but since that is "C" it's dubious registers are used for
long time storage (contrary to what ASM code does profusely, at least mine).

if the master environment would be simply
'data' with some internal variable pointing to it, the master
environment would be obviously swapped out (or located at ffff:xxxx)

but of course you knew that

tom

[Bertho Grandpied]

On Mon, 8 Jul 2013 16:29:14 +0200

Post by Tom Ehlert

Post by Bertho Grandpied
I may sound harsh, but being accused of ignorance by
more ignorant is the only word of excuse I will utter.

this 'more' makes me think that you should prove your competence

first

I agree. The above quoted phrase was a late minute, unfortunate
addition to my mail, that was meant jokingly - but now on reading it
again I can see it does not reflect my true purpose, which has been to
put an end to respective sarcasm and ire. Sincerely ! That "more" was meant to be "no less", as in, ok, we're all ignorants, let's live with the fact and do our best to learn from one another. Oh, well...

Post by Tom Ehlert

Post by Bertho Grandpied
Let's forget /name calling/ ? If you agree, I'm your man

Concerning the subject - locating the main env immediately above
the small 3 K ? block of FreeCom's PSP, I honestly cannot discern
whether you are seriously in want of explanations or only asking
in jest. It is not complicated, at least from an ASM programmer's
perspective where you can organise your modules at will.

Or were you questioning the feasibility from the perspective of
or a complex modular 'C' language program ?
Then the detailed how is beyond my limited competence,
I /suppose/ it might be harder, possibly requiring compiler-dependent
variations and /some/ amount of 'glue' coded in ASM.

Coding details are not to derail this mail list, you'll agree.
We might dig down slightly deeper in private email, or on the
developers' list if it's of interest to the community.

Post by Tom Ehlert

Post by Bertho Grandpied
not rocket science.

But as they say, the devil is in the details ..

Post by Tom Ehlert
it looks like a real experienced programmer (you) has to show us
confused youngsters how it should be done!
I'm deep impressed. we have a black belt grandmaster on
visit. unfortunately too
busy to work, but we are so happy to receive good advice ;)

I can take sarcasm still :=)

Post by Tom Ehlert
There are even other enhancements that could be made in

the same domain,

Post by Tom Ehlert
an easy one is to give back the /initial/ environment

Retiring this as already done by FreeCOM (yeah!)

I have no idea why MS-Command doesn't , 4DOS neither.
In particular, when one of those other Command's is the primary
shell in MS-DOS 6+, the intial env block inherited from config.sys,
is retained which causes incompatibilities with some older DOS
programs.

Bye !

[Tom Ehlert]

Post by Bertho Grandpied

Post by Tom Ehlert

Post by Bertho Grandpied
I may sound harsh, but being accused of ignorance by
more ignorant is the only word of excuse I will utter.

this 'more' makes me think that you should prove your competence

first
I agree. The above quoted phrase was a late minute, unfortunate
addition to my mail, that was meant jokingly - but now on reading it
again I can see it does not reflect my true purpose, which has been to
put an end to respective sarcasm and ire. Sincerely ! That "more"
was meant to be "no less", as in, ok, we're all ignorants, let's
live with the fact and do our best to learn from one another. Oh, well...

Post by Tom Ehlert

Post by Bertho Grandpied
Let's forget /name calling/ ? If you agree, I'm your man

apologies accepted.

Post by Bertho Grandpied
Concerning the subject - locating the main env immediately above
the small 3 K ? block of FreeCom's PSP,
I honestly cannot discern
whether you are seriously in want of explanations or only asking
in jest. It is not complicated, at least from an ASM programmer's
perspective where you can organise your modules at will.
Or were you questioning the feasibility from the perspective of
or a complex modular 'C' language program ?
Then the detailed how is beyond my limited competence,
I /suppose/ it might be harder, possibly requiring compiler-dependent
variations and /some/ amount of 'glue' coded in ASM.

the bizarre design decision by a confused programmer was made because
a single 'load as high as possible' takes care of all the trouble.
(there's enough necessary trouble left to swap command in/out of XMS...)
I call this 'intelligent design', and so far there were no adverse
effects reported.

sure your particular problem could be solved by freecom, but nobody
will spend time on this.

it can be solved as well or better by a small external utility; just have
the first line in autoexec.bat be ENVLOW.EXE.
and moving program code around is probbaly easier in ASM then in C

even better: you can do it yourself without learning C ;)

Post by Bertho Grandpied

Post by Tom Ehlert

Post by Bertho Grandpied
not rocket science.

memory juggling in freecom was advanced engineering; in kernel.sys it
*was* rocket science.

Tom

[Bertho Grandpied]

Hi again, Tom...

Post by Tom Ehlert
that's how 'edit the parent/master environment' utilities work(ed),
it's 'documented undocumented' behaviour, and FreeCOM behaves
and has to behave) the same way.

It's a good point, but it was not written in stone.
Thus it is nice - for our purpose - that FreeCM taking compatibility to heart, adheres to this one :=)

It will be nicer when compatibility applies to the placement of the ENV, too !

Post by Tom Ehlert
'data' with some internal variable pointing to it, the master
environment would be obviously swapped out (or located at ffff:xxxx)

I'm not certain the HMA would be an adequate container for, say, a
a master ENV which could be 32 kilobytes large, say...

See you...

[Bertho Grandpied]

Post by Tom Ehlert
sure your particular problem could be solved by freecom, but nobody
will spend time on this.

Still, Rugxulo, I think, has suggested one could ping Bart Oldemann with the question.

Now, after you have pointed out very plausibly that FreeCOM does not
hold "hidden" pointers to the master ENV, I must admit I do not see the
point originally made as crucial any more.

Post by Tom Ehlert
it can be solved as well or better by a small external utility; just have
the first line in autoexec.bat be ENVLOW.EXE.

Is that found in the FreeDOS distro ? Don't bother, I'll g00gle it...

This list's Macej brewed his own tiny, simple mover and sent a copy
for me to play with (thanks Macej).. It's amusing, but needs to be run several times in a row to get a chance to relocate the ENV properly :=)

Post by Tom Ehlert
and moving program code around is probably easier in ASM then in C

Undoubtably

Post by Tom Ehlert
even better: you can do it yourself without learning C ;)

Great relief, that! C was not around when I educated myself to
programming in the 50s - early 60s,.by the way.

I intend to do mine indeed - unless the envlow you mentionned
is doing the job.

Macej's is good as a proof of concept but does not quite do what I want done, which is leave no hole after it.

DOS memory ought not to look like Swiss cheese (gruyere)....

Hmm, question may be considered closed for the moment being.

Cheers