Finding Imported Modules

[gray]

From the Astrobe docs:

The editor, compiler, linker and builder first search the current folder when trying to locate imported source code, symbol and object files. They then search each of the folders listed in the current configuration’s Library Pathnames textbox.

Consider this test set-up:

Code: Select all

+ C:\Test
  + lib
    + Main.mod
    + M0.mod
    + M1.mod
  + prog
    + M1.mod
    + P.mod

Code:

Code: Select all

MODULE P;
  IMPORT Main, M0;
END P.

MODULE Main;
END Main.

MODULE M0;
  IMPORT M1;
END M0.

MODULE M1;
  (* same in 'lib' and 'prog' *)
END M1.

Library search path:

Code: Select all

C:\Test\lib

Buidling with all directories "clean" of any symbol and object files:

Code: Select all

Astrobe for RP2350 Builder v10.0.1
Builder Phase 1: Checking P.mod and imported modules...

Checking Module Main
  Imports:
  Folder: C:\Test\lib
  Status: Missing Main.arm file
Checking Module M1
  Imports:
  Folder: C:\Test\prog
  Status: Missing M1.arm file
Checking Module M0
  Imports:
    M1
  Folder: C:\Test\lib
  Status: Missing M0.arm file
Checking Module P
  Imports:
    Main
    M0
  Folder: C:\Test\prog
  Status: Missing P.arm file
------------------

Builder Phase 2: Compiling missing / outdated modules...

  compiling C:\Test\lib\Main.mod
new symbol file;  code generated = 8 bytes, data = 0 bytes, entries = 0, lines = 2, msecs = 11
------------------
  compiling C:\Test\prog\M1.mod
new symbol file;  code generated = 8 bytes, data = 0 bytes, entries = 0, lines = 2, msecs = 1
------------------
  compiling C:\Test\lib\M0.mod

  Line  Col
     2   13 Error: import not available
------------------
2 modules compiled, lines = 6, msecs = 13

That is, a compiled M1.mod in the same directory as M0.mod is not found, unsurprisingly, as M1.mod in 'prog' was compiled, not the one in 'lib', and the search path does not lead to 'prog'.

Running with this search path, again with clean directories:

Code: Select all

C:\Test\prog
C:\Test\lib

results in sucessful compilation and linking (only linker output shown):

Code: Select all

Astrobe for RP2350 Linker v10.0.1
module Main  6E69614DH
filename C:\Test\lib\Main.arm
module P  00000000H
filename C:\Test\prog\P.arm
  importing M0  0300304DH     0     0
module M0  0300304DH
filename C:\Test\lib\M0.arm
  importing M1  0300314DH     0     0
module M1  0300314DH
filename C:\Test\prog\M1.arm
  linking Main                 8 bytes
  linking M1                   8 bytes
  linking M0                   8 bytes
  linking P                    8 bytes

Unless I miss something here, this does not conform the documentation: in both cases, M1.mod in 'lib' should have been picked, since the imported module in the same directory as the importer should take precedence.

Now, I would prefer if the observed behaviour were the intended one, as it allows to easily replace library modules with custom ones. Please clarify, thanks.

[cfbsoftware]

The 'current folder' is the folder which contains the source code of the target of the operation . When linking and building the target is the module which imports Main. When compiling a module, the 'current folder' is the folder which contains the source code of the module being compiled.

The system is designed so that all of the files created when you compile a module (*.smb, *.arm etc.) are located in the same folder as the source code of that module. Similarly, all of the files that are created when you link a main module (*.bin, *.uf2 etc.) are located in the same folder as the source code of that module. However, the system allows you to reference files in other folders in a hierarchical fashion.

A constraint of the system is that it is an error if you attempt to link a module which references two or more different versions of a module with the same name. Hence, it is unwise to design a system which has two modules with the same name in the same hierarchy as in your example. For example, the following code results in the error: 'Module M1 - wrong version':

-----------------------------------------
Test Folder:
MODULE P;
IMPORT Main, M0, M1;
VAR
p: INTEGER;
BEGIN
p := M1.prog
END P.
-----------------------------------------
Lib Folder:
MODULE Main;
END Main.

MODULE M0;
IMPORT M1;
END M0.

MODULE M1;
VAR
lib*: INTEGER;
(* Lib Version *)
END M1.
----------------------------------------
Prog Folder:
MODULE M1;
VAR prog*: INTEGER;
(* Prog version *)
END M1.[/code]
-----------------------------------------

[gray]

For example, the following code results in the error: 'Module M1 - wrong version':

Well, that depends. Compiling and linking your example code works fine, if I

1) start with clean directories;
2) compile and link from the "top", ie. P.mod.
3) repeatedly recompile and link from P.mod, ie. without clean directories, as during development.

No wrong version issues. M0.mod compiles with reference to M1.mod in 'prog', and the compilation results are in the same directory as the source. M1.mod in 'lib' does not get compiled.

It's only when I compile M1.mod in 'lib', and then compile M0.mod "locally", ie. with 'lib' as "current folder", I will run into the wrong version issue when attempting to build again from P.mod, since M0.mod is now compiled "against" M1.mod in 'lib', due to the "current folder" rule.

The problem I want to solve is to replace specific library modules, such as provided by Astrobe or Oberon RTK, with program/project-specific custom ones. Since in this case I never need to compile not replaced modules in the libray locally, this would work as is, but it's brittle, in particular if the library is used by different programs. I'll think of something. :) Probably based on program-local sub-sets of the library (actually used, sans the custom ones), supported by a tool for update and maintenance. Segregating library modules into "can be customised" directories in the library structure is not an option in general.

[gray]

Mulling over this some more, in the light of the documentation, and all you said in this thread and in Relative Search Path makes me think that I should not be able to sucessfully build the test program as described in my post above. That is, with M0 compiled with reference to M1 in 'prog' (verified with the M1 module key and the reference to it in M0.arm), and M1 in 'lib' not being compiled.

[cfbsoftware]

In your original example M1 in lib is identical to M1 in prog which is not a realistic scenario. It wouldn't matter which one is picked up by the linker. In a valid example of what I believe you are trying to test / demonstrate, the public interface of the modules (i.e. the exported items) would need to be different.

[gray]

As said, I am building your test code, ie. with two modules M1 with different public interfaces. As long as I don't compile M0 directly, I can build P successfully, with M1 in 'lib' never being compiled. I attach the test code (Test2.zip).

To take the test one step further, if I change M0 to

Code: Select all

MODULE M0;
  IMPORT M1;
  VAR p: INTEGER;
BEGIN
  p := M1.lib (* M1 in 'lib' together with M0 *)
END M0.

with rest of the code unchanged, I get

Code: Select all

Astrobe for RP2350 Builder v10.0.1
Builder Phase 1: Checking P.mod and imported modules...

Checking Module Main
  Imports:
  Folder: C:\Astrobe\Test4\lib
  Status: Missing Main.arm file
Checking Module M1
  Imports:
  Folder: C:\Astrobe\Test4\prog
  Status: Missing M1.arm file
Checking Module M0
  Imports:
    M1
  Folder: C:\Astrobe\Test4\lib
  Status: Missing M0.arm file
Checking Module P
  Imports:
    Main
    M0
    M1
  Folder: C:\Astrobe\Test4
  Status: Missing P.arm file
------------------

Builder Phase 2: Compiling missing / outdated modules...

  compiling Main
new symbol file;  code generated = 8 bytes, data = 0 bytes, entries = 0, lines = 2, msecs = 2
------------------
  compiling M1
new symbol file;  code generated = 8 bytes, data = 4 bytes, entries = 1, lines = 3, msecs = 1
------------------
  compiling M0

  Line  Col
     6    4 Error: undef

That is, M0 is being compiled with reference to M1 in 'prog', not M1 in 'lib'. Here, the problem does not manifest with the linker due to different module versions, but already in compilation. See attached Test4.zip.

[cfbsoftware]

I'm sorry - you've lost me. Are you now saying it is all working as it should?

If not and problems only arise if there are two or more modules with the same name in the current list of search paths I can try and modify the system to report that as an error.

[gray]

Allow me summarise.

1) The following is the test code you have provided. I have only changed VARs to CONST in both M1.mod to easier check in the assembly file which module is actually imported. But importantly, both M1.mod have a different public interface.

Structure:

Code: Select all

+ Test2
  + P1.mod
  + prog
    + M1.mod
  + lib
    + M0.mod
    + M1.mod

Code:

Code: Select all

MODULE P;
  IMPORT Main, M0, M1;
  VAR p: INTEGER;
BEGIN
  p := M1.prog
END P.

MODULE M0;
  IMPORT M1;
END M0.

MODULE M1; (* in 'lib' *)
  CONST lib* = 42;
END M1.

MODULE M1; (* in 'prog' *)
  CONST prog* = 13;
END M1.

MODULE Main;
END Main.

Search path:

Code: Select all

C:\Astrobe\Test2\prog
C:\Astrobe\Test2\lib

2) You said that "the following code results in the error: 'Module M1 - wrong version".

3) I have demonstrated that this is not necessarily the case, but that the program compiles and links without problems, if we start building P.mod from directories void of any symbol and object files.

Code: Select all

Astrobe for RP2350 Builder v10.0.1
Builder Phase 1: Checking P.mod and imported modules...

Checking Module Main
  Imports:
  Folder: C:\Astrobe\Test2\lib
  Status: Missing Main.arm file
Checking Module M1
  Imports:
  Folder: C:\Astrobe\Test2\prog
  Status: Missing M1.arm file
Checking Module M0
  Imports:
    M1
  Folder: C:\Astrobe\Test2\lib
  Status: Missing M0.arm file
Checking Module P
  Imports:
    Main
    M0
    M1
  Folder: C:\Astrobe\Test2
  Status: Missing P.arm file
------------------

Builder Phase 2: Compiling missing / outdated modules...

  compiling Main
new symbol file;  code generated = 8 bytes, data = 0 bytes, entries = 0, lines = 2, msecs = 2
------------------
  compiling M1
new symbol file;  code generated = 8 bytes, data = 0 bytes, entries = 0, lines = 3, msecs = 3
------------------
  compiling M0

  Line  Col
     3    4 Warning: M1 is not used
new symbol file;  code generated = 8 bytes, data = 0 bytes, entries = 0, lines = 3, msecs = 2
------------------
  compiling P

  Line  Col
     6    4 Warning: M0 is not used
new symbol file;  code generated = 20 bytes, data = 4 bytes, entries = 0, lines = 6, msecs = 15
------------------
4 modules compiled, lines = 14, msecs = 22
------------------

Builder Phase 3: Linking all modules...

  linking Main                 8 bytes
  linking M1                   8 bytes
  linking M0                   8 bytes
  linking P                   20 bytes

Stack: 2002FFF8H, Entry: 1000036DH
Created P.uf2

Max Extension Level:     4

Total Code Size:        620 bytes
Total Resource Size:    216 bytes
Total ROM Used:         836 bytes
Total Data Size:          4 bytes

Configuration:     RP2350 test
Configuration ID:       0
Data Range:        20000000H  20030000H
Code Range:        10000100H  10200000H
Heap Start:        20000200H
Heap Limit:        00000000H
Linking completed

4) As I understand, the build system

a) performs separate compilations for all modules,
b) looking for imports first in the directory of each module being compiled, then following the library search path,
c) performing the separate compilations in the correct order so that each module being compiled finds all its imports already compiled.

If that is true, the above successful build should not be possible. I believe it points to a flaw in the build system. Namely, for compiling M0.mod, M1.mod in 'lib' should have been recognised as correct import and compiled. Then the linker would point out the module version clash between M1.mod in 'prog' as imported by P.mod, and M1.mod in 'lib' as imported by M0.mod. In the test case, M1.mod in 'lib' is not detected by the build system.

5) If we simulate the relevant part of the (IMO) correct build process, and directly first compile M1.mod in 'lib' and then M0.mod, and then build P.mod we get the expected module version error message.

This is an admittedly convoluted test case, but then again, good test cases often are. :)

***

Regarding not allowing several modules with the same name in the same search path (including the implicit current module directories): please do not change the current concept! While each compiled and linked program must only contain one module with a specific name, the search path can allow for more -- as long as only one is picked in the build process (static vs. dynamic set of modules).

In fact, allowing this set-up is a feature, not a defect. All library modules that are not imported by any other library module can be easily replaced by a program-specific one. The main example is -- no pun intended -- module Main.mod. I place my Main.mod in the program directory, and voila, my module takes precedence, without any need to remove it from the library. Maybe I need a different clock frequency, or a different UART baudrate, or additional initialisations. I use this feature for quite a few example programs. In Oberon RTK there are also other modules of equivalent nature as Main.mod. We can selectively be "unwise". :)

The above counter test example notwithstanding, Astrobe provides suffient protection against "double" modules already.

[cfbsoftware]

OK - thanks. A couple of points:

2) You said that "the following code results in the error: 'Module M1 - wrong version".

Correction - I should have said:
"the following code can result in the error: 'Module M1 - wrong version"

4) As I understand, the build system

a) performs separate compilations for all modules,

a) performs separate compilations for modules that need to be compiled

Note that when using relative paths the linker paths can be different from the compilation paths. If you do have duplicate modules you can minimise problems by always deleting all previously generated files in all of the paths before performing a build.