forked from spack/spack
-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathspec.py
More file actions
5551 lines (4585 loc) · 211 KB
/
spec.py
File metadata and controls
5551 lines (4585 loc) · 211 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
"""
Spack allows very fine-grained control over how packages are installed and
over how they are built and configured. To make this easy, it has its own
syntax for declaring a dependence. We call a descriptor of a particular
package configuration a "spec".
The syntax looks like this:
.. code-block:: sh
$ spack install mpileaks ^openmpi @1.2:1.4 +debug %intel @12.1 target=zen
0 1 2 3 4 5 6
The first part of this is the command, 'spack install'. The rest of the
line is a spec for a particular installation of the mpileaks package.
0. The package to install
1. A dependency of the package, prefixed by ^
2. A version descriptor for the package. This can either be a specific
version, like "1.2", or it can be a range of versions, e.g. "1.2:1.4".
If multiple specific versions or multiple ranges are acceptable, they
can be separated by commas, e.g. if a package will only build with
versions 1.0, 1.2-1.4, and 1.6-1.8 of mavpich, you could say:
depends_on("mvapich@1.0,1.2:1.4,1.6:1.8")
3. A compile-time variant of the package. If you need openmpi to be
built in debug mode for your package to work, you can require it by
adding +debug to the openmpi spec when you depend on it. If you do
NOT want the debug option to be enabled, then replace this with -debug.
4. The name of the compiler to build with.
5. The versions of the compiler to build with. Note that the identifier
for a compiler version is the same '@' that is used for a package version.
A version list denoted by '@' is associated with the compiler only if
if it comes immediately after the compiler name. Otherwise it will be
associated with the current package spec.
6. The architecture to build with. This is needed on machines where
cross-compilation is required
Here is the EBNF grammar for a spec::
spec-list = { spec [ dep-list ] }
dep_list = { ^ spec }
spec = id [ options ]
options = { @version-list | +variant | -variant | ~variant |
%compiler | arch=architecture | [ flag ]=value}
flag = { cflags | cxxflags | fcflags | fflags | cppflags |
ldflags | ldlibs }
variant = id
architecture = id
compiler = id [ version-list ]
version-list = version [ { , version } ]
version = id | id: | :id | id:id
id = [A-Za-z0-9_][A-Za-z0-9_.-]*
Identifiers using the <name>=<value> command, such as architectures and
compiler flags, require a space before the name.
There is one context-sensitive part: ids in versions may contain '.', while
other ids may not.
There is one ambiguity: since '-' is allowed in an id, you need to put
whitespace space before -variant for it to be tokenized properly. You can
either use whitespace, or you can just use ~variant since it means the same
thing. Spack uses ~variant in directory names and in the canonical form of
specs to avoid ambiguity. Both are provided because ~ can cause shell
expansion when it is the first character in an id typed on the command line.
"""
import collections
import itertools
import operator
import os
import re
import sys
import warnings
import ruamel.yaml as yaml
import six
import llnl.util.filesystem as fs
import llnl.util.lang as lang
import llnl.util.tty as tty
import llnl.util.tty.color as clr
from llnl.util.compat import Mapping
import spack.compiler
import spack.compilers
import spack.config
import spack.dependency as dp
import spack.error
import spack.hash_types as ht
import spack.parse
import spack.paths
import spack.platforms
import spack.provider_index
import spack.repo
import spack.solver
import spack.store
import spack.target
import spack.util.crypto
import spack.util.executable
import spack.util.hash
import spack.util.module_cmd as md
import spack.util.path as pth
import spack.util.prefix
import spack.util.spack_json as sjson
import spack.util.spack_yaml as syaml
import spack.util.string
import spack.variant as vt
import spack.version as vn
__all__ = [
'CompilerSpec',
'Spec',
'SpecParser',
'parse',
'SpecParseError',
'DuplicateDependencyError',
'DuplicateCompilerSpecError',
'UnsupportedCompilerError',
'DuplicateArchitectureError',
'InconsistentSpecError',
'InvalidDependencyError',
'NoProviderError',
'MultipleProviderError',
'UnsatisfiableSpecNameError',
'UnsatisfiableVersionSpecError',
'UnsatisfiableCompilerSpecError',
'UnsatisfiableCompilerFlagSpecError',
'UnsatisfiableArchitectureSpecError',
'UnsatisfiableProviderSpecError',
'UnsatisfiableDependencySpecError',
'AmbiguousHashError',
'InvalidHashError',
'NoSuchHashError',
'RedundantSpecError',
'SpecDeprecatedError',
]
is_windows = sys.platform == 'win32'
#: Valid pattern for an identifier in Spack
identifier_re = r'\w[\w-]*'
compiler_color = '@g' #: color for highlighting compilers
version_color = '@c' #: color for highlighting versions
architecture_color = '@m' #: color for highlighting architectures
enabled_variant_color = '@B' #: color for highlighting enabled variants
disabled_variant_color = '@r' #: color for highlighting disabled varaints
dependency_color = '@.' #: color for highlighting dependencies
hash_color = '@K' #: color for highlighting package hashes
#: This map determines the coloring of specs when using color output.
#: We make the fields different colors to enhance readability.
#: See llnl.util.tty.color for descriptions of the color codes.
color_formats = {'%': compiler_color,
'@': version_color,
'=': architecture_color,
'+': enabled_variant_color,
'~': disabled_variant_color,
'^': dependency_color,
'#': hash_color}
#: Regex used for splitting by spec field separators.
#: These need to be escaped to avoid metacharacters in
#: ``color_formats.keys()``.
_separators = '[\\%s]' % '\\'.join(color_formats.keys())
#: Versionlist constant so we don't have to build a list
#: every time we call str()
_any_version = vn.VersionList([':'])
default_format = '{name}{@version}'
default_format += '{%compiler.name}{@compiler.version}{compiler_flags}'
default_format += '{variants}{arch=architecture}'
#: specfile format version. Must increase monotonically
specfile_format_version = 2
def colorize_spec(spec):
"""Returns a spec colorized according to the colors specified in
color_formats."""
class insert_color:
def __init__(self):
self.last = None
def __call__(self, match):
# ignore compiler versions (color same as compiler)
sep = match.group(0)
if self.last == '%' and sep == '@':
return clr.cescape(sep)
self.last = sep
return '%s%s' % (color_formats[sep], clr.cescape(sep))
return clr.colorize(re.sub(_separators, insert_color(), str(spec)) + '@.')
@lang.lazy_lexicographic_ordering
class ArchSpec(object):
"""Aggregate the target platform, the operating system and the target
microarchitecture into an architecture spec..
"""
@staticmethod
def _return_arch(os_tag, target_tag):
platform = spack.platforms.host()
default_os = platform.operating_system(os_tag)
default_target = platform.target(target_tag)
arch_tuple = str(platform), str(default_os), str(default_target)
return ArchSpec(arch_tuple)
@staticmethod
def default_arch():
"""Return the default architecture"""
return ArchSpec._return_arch('default_os', 'default_target')
@staticmethod
def frontend_arch():
"""Return the frontend architecture"""
return ArchSpec._return_arch('frontend', 'frontend')
def __init__(self, spec_or_platform_tuple=(None, None, None)):
""" Architecture specification a package should be built with.
Each ArchSpec is comprised of three elements: a platform (e.g. Linux),
an OS (e.g. RHEL6), and a target (e.g. x86_64).
Args:
spec_or_platform_tuple (ArchSpec or str or tuple): if an ArchSpec
is passed it will be duplicated into the new instance.
Otherwise information on platform, OS and target should be
passed in either as a spec string or as a tuple.
"""
# If the argument to __init__ is a spec string, parse it
# and construct an ArchSpec
def _string_or_none(s):
if s and s != 'None':
return str(s)
return None
# If another instance of ArchSpec was passed, duplicate it
if isinstance(spec_or_platform_tuple, ArchSpec):
other = spec_or_platform_tuple
platform_tuple = other.platform, other.os, other.target
elif isinstance(spec_or_platform_tuple, (six.string_types, tuple)):
spec_fields = spec_or_platform_tuple
# Normalize the string to a tuple
if isinstance(spec_or_platform_tuple, six.string_types):
spec_fields = spec_or_platform_tuple.split("-")
if len(spec_fields) != 3:
msg = 'cannot construct an ArchSpec from {0!s}'
raise ValueError(msg.format(spec_or_platform_tuple))
platform, operating_system, target = spec_fields
platform_tuple = (
_string_or_none(platform),
_string_or_none(operating_system),
target
)
self.platform, self.os, self.target = platform_tuple
def _autospec(self, spec_like):
if isinstance(spec_like, ArchSpec):
return spec_like
return ArchSpec(spec_like)
def _cmp_iter(self):
yield self.platform
yield self.os
yield self.target
@property
def platform(self):
"""The platform of the architecture."""
return self._platform
@platform.setter
def platform(self, value):
# The platform of the architecture spec will be verified as a
# supported Spack platform before it's set to ensure all specs
# refer to valid platforms.
value = str(value) if value is not None else None
self._platform = value
@property
def os(self):
"""The OS of this ArchSpec."""
return self._os
@os.setter
def os(self, value):
# The OS of the architecture spec will update the platform field
# if the OS is set to one of the reserved OS types so that the
# default OS type can be resolved. Since the reserved OS
# information is only available for the host machine, the platform
# will assumed to be the host machine's platform.
value = str(value) if value is not None else None
if value in spack.platforms.Platform.reserved_oss:
curr_platform = str(spack.platforms.host())
self.platform = self.platform or curr_platform
if self.platform != curr_platform:
raise ValueError(
"Can't set arch spec OS to reserved value '%s' when the "
"arch platform (%s) isn't the current platform (%s)" %
(value, self.platform, curr_platform))
spec_platform = spack.platforms.by_name(self.platform)
value = str(spec_platform.operating_system(value))
self._os = value
@property
def target(self):
"""The target of the architecture."""
return self._target
@target.setter
def target(self, value):
# The target of the architecture spec will update the platform field
# if the target is set to one of the reserved target types so that
# the default target type can be resolved. Since the reserved target
# information is only available for the host machine, the platform
# will assumed to be the host machine's platform.
def target_or_none(t):
if isinstance(t, spack.target.Target):
return t
if t and t != 'None':
return spack.target.Target(t)
return None
value = target_or_none(value)
if str(value) in spack.platforms.Platform.reserved_targets:
curr_platform = str(spack.platforms.host())
self.platform = self.platform or curr_platform
if self.platform != curr_platform:
raise ValueError(
"Can't set arch spec target to reserved value '%s' when "
"the arch platform (%s) isn't the current platform (%s)" %
(value, self.platform, curr_platform))
spec_platform = spack.platforms.by_name(self.platform)
value = spec_platform.target(value)
self._target = value
def satisfies(self, other, strict=False):
"""Predicate to check if this spec satisfies a constraint.
Args:
other (ArchSpec or str): constraint on the current instance
strict (bool): if ``False`` the function checks if the current
instance *might* eventually satisfy the constraint. If
``True`` it check if the constraint is satisfied right now.
Returns:
True if the constraint is satisfied, False otherwise.
"""
other = self._autospec(other)
# Check platform and os
for attribute in ('platform', 'os'):
other_attribute = getattr(other, attribute)
self_attribute = getattr(self, attribute)
if strict or self.concrete:
if other_attribute and self_attribute != other_attribute:
return False
else:
if other_attribute and self_attribute and \
self_attribute != other_attribute:
return False
# Check target
return self.target_satisfies(other, strict=strict)
def target_satisfies(self, other, strict):
need_to_check = bool(other.target) if strict or self.concrete \
else bool(other.target and self.target)
# If there's no need to check we are fine
if not need_to_check:
return True
# self is not concrete, but other_target is there and strict=True
if self.target is None:
return False
return bool(self.target_intersection(other))
def target_constrain(self, other):
if not other.target_satisfies(self, strict=False):
raise UnsatisfiableArchitectureSpecError(self, other)
if self.target_concrete:
return False
elif other.target_concrete:
self.target = other.target
return True
# Compute the intersection of every combination of ranges in the lists
results = self.target_intersection(other)
# Do we need to dedupe here?
self.target = ','.join(results)
def target_intersection(self, other):
results = []
if not self.target or not other.target:
return results
for s_target_range in str(self.target).split(','):
s_min, s_sep, s_max = s_target_range.partition(':')
for o_target_range in str(other.target).split(','):
o_min, o_sep, o_max = o_target_range.partition(':')
if not s_sep:
# s_target_range is a concrete target
# get a microarchitecture reference for at least one side
# of each comparison so we can use archspec comparators
s_comp = spack.target.Target(s_min).microarchitecture
if not o_sep:
if s_min == o_min:
results.append(s_min)
elif (not o_min or s_comp >= o_min) and (
not o_max or s_comp <= o_max):
results.append(s_min)
elif not o_sep:
# "cast" to microarchitecture
o_comp = spack.target.Target(o_min).microarchitecture
if (not s_min or o_comp >= s_min) and (
not s_max or o_comp <= s_max):
results.append(o_min)
else:
# Take intersection of two ranges
# Lots of comparisons needed
_s_min = spack.target.Target(s_min).microarchitecture
_s_max = spack.target.Target(s_max).microarchitecture
_o_min = spack.target.Target(o_min).microarchitecture
_o_max = spack.target.Target(o_max).microarchitecture
n_min = s_min if _s_min >= _o_min else o_min
n_max = s_max if _s_max <= _o_max else o_max
_n_min = spack.target.Target(n_min).microarchitecture
_n_max = spack.target.Target(n_max).microarchitecture
if _n_min == _n_max:
results.append(n_min)
elif not n_min or not n_max or _n_min < _n_max:
results.append('%s:%s' % (n_min, n_max))
return results
def constrain(self, other):
"""Projects all architecture fields that are specified in the given
spec onto the instance spec if they're missing from the instance
spec.
This will only work if the two specs are compatible.
Args:
other (ArchSpec or str): constraints to be added
Returns:
True if the current instance was constrained, False otherwise.
"""
other = self._autospec(other)
if not other.satisfies(self):
raise UnsatisfiableArchitectureSpecError(other, self)
constrained = False
for attr in ('platform', 'os'):
svalue, ovalue = getattr(self, attr), getattr(other, attr)
if svalue is None and ovalue is not None:
setattr(self, attr, ovalue)
constrained = True
self.target_constrain(other)
return constrained
def copy(self):
"""Copy the current instance and returns the clone."""
return ArchSpec(self)
@property
def concrete(self):
"""True if the spec is concrete, False otherwise"""
# return all(v for k, v in six.iteritems(self.to_cmp_dict()))
return (self.platform and self.os and self.target and
self.target_concrete)
@property
def target_concrete(self):
"""True if the target is not a range or list."""
return ':' not in str(self.target) and ',' not in str(self.target)
def to_dict(self):
d = syaml.syaml_dict([
('platform', self.platform),
('platform_os', self.os),
('target', self.target.to_dict_or_value())])
return syaml.syaml_dict([('arch', d)])
@staticmethod
def from_dict(d):
"""Import an ArchSpec from raw YAML/JSON data"""
arch = d['arch']
target = spack.target.Target.from_dict_or_value(arch['target'])
return ArchSpec((arch['platform'], arch['platform_os'], target))
def __str__(self):
return "%s-%s-%s" % (self.platform, self.os, self.target)
def __repr__(self):
fmt = 'ArchSpec(({0.platform!r}, {0.os!r}, {1!r}))'
return fmt.format(self, str(self.target))
def __contains__(self, string):
return string in str(self) or string in self.target
@lang.lazy_lexicographic_ordering
class CompilerSpec(object):
"""The CompilerSpec field represents the compiler or range of compiler
versions that a package should be built with. CompilerSpecs have a
name and a version list. """
def __init__(self, *args):
nargs = len(args)
if nargs == 1:
arg = args[0]
# If there is one argument, it's either another CompilerSpec
# to copy or a string to parse
if isinstance(arg, six.string_types):
c = SpecParser().parse_compiler(arg)
self.name = c.name
self.versions = c.versions
elif isinstance(arg, CompilerSpec):
self.name = arg.name
self.versions = arg.versions.copy()
else:
raise TypeError(
"Can only build CompilerSpec from string or " +
"CompilerSpec. Found %s" % type(arg))
elif nargs == 2:
name, version = args
self.name = name
self.versions = vn.VersionList()
self.versions.add(vn.ver(version))
else:
raise TypeError(
"__init__ takes 1 or 2 arguments. (%d given)" % nargs)
def _add_versions(self, version_list):
# If it already has a non-trivial version list, this is an error
if self.versions and self.versions != vn.VersionList(':'):
# Note: This may be impossible to reach by the current parser
# Keeping it in case the implementation changes.
raise MultipleVersionError(
'A spec cannot contain multiple version signifiers.'
' Use a version list instead.')
self.versions = vn.VersionList()
for version in version_list:
self.versions.add(version)
def _autospec(self, compiler_spec_like):
if isinstance(compiler_spec_like, CompilerSpec):
return compiler_spec_like
return CompilerSpec(compiler_spec_like)
def satisfies(self, other, strict=False):
other = self._autospec(other)
return (self.name == other.name and
self.versions.satisfies(other.versions, strict=strict))
def constrain(self, other):
"""Intersect self's versions with other.
Return whether the CompilerSpec changed.
"""
other = self._autospec(other)
# ensure that other will actually constrain this spec.
if not other.satisfies(self):
raise UnsatisfiableCompilerSpecError(other, self)
return self.versions.intersect(other.versions)
@property
def concrete(self):
"""A CompilerSpec is concrete if its versions are concrete and there
is an available compiler with the right version."""
return self.versions.concrete
@property
def version(self):
if not self.concrete:
raise spack.error.SpecError("Spec is not concrete: " + str(self))
return self.versions[0]
def copy(self):
clone = CompilerSpec.__new__(CompilerSpec)
clone.name = self.name
clone.versions = self.versions.copy()
return clone
def _cmp_iter(self):
yield self.name
yield self.versions
def to_dict(self):
d = syaml.syaml_dict([('name', self.name)])
d.update(self.versions.to_dict())
return syaml.syaml_dict([('compiler', d)])
@staticmethod
def from_dict(d):
d = d['compiler']
return CompilerSpec(d['name'], vn.VersionList.from_dict(d))
def __str__(self):
out = self.name
if self.versions and self.versions != _any_version:
vlist = ",".join(str(v) for v in self.versions)
out += "@%s" % vlist
return out
def __repr__(self):
return str(self)
@lang.lazy_lexicographic_ordering
class DependencySpec(object):
"""DependencySpecs connect two nodes in the DAG, and contain deptypes.
Dependencies can be one (or more) of several types:
- build: needs to be in the PATH at build time.
- link: is linked to and added to compiler flags.
- run: needs to be in the PATH for the package to run.
Fields:
- spec: Spec depended on by parent.
- parent: Spec that depends on `spec`.
- deptypes: list of strings, representing dependency relationships.
"""
def __init__(self, parent, spec, deptypes):
self.parent = parent
self.spec = spec
self.deptypes = dp.canonical_deptype(deptypes)
def update_deptypes(self, deptypes):
deptypes = set(deptypes)
deptypes.update(self.deptypes)
deptypes = tuple(sorted(deptypes))
changed = self.deptypes != deptypes
self.deptypes = deptypes
return changed
def copy(self):
return DependencySpec(self.parent, self.spec, self.deptypes)
def add_type(self, type):
self.deptypes = dp.canonical_deptype(
self.deptypes + dp.canonical_deptype(type)
)
def _cmp_iter(self):
yield self.parent.name if self.parent else None
yield self.spec.name if self.spec else None
yield self.deptypes
def __str__(self):
return "%s %s--> %s" % (self.parent.name if self.parent else None,
self.deptypes,
self.spec.name if self.spec else None)
def canonical(self):
return self.parent.dag_hash(), self.spec.dag_hash(), self.deptypes
_valid_compiler_flags = [
'cflags', 'cxxflags', 'fflags', 'ldflags', 'ldlibs', 'cppflags']
class FlagMap(lang.HashableMap):
def __init__(self, spec):
super(FlagMap, self).__init__()
self.spec = spec
def satisfies(self, other, strict=False):
if strict or (self.spec and self.spec._concrete):
return all(f in self and set(self[f]) == set(other[f])
for f in other)
else:
return all(set(self[f]) == set(other[f])
for f in other if (other[f] != [] and f in self))
def constrain(self, other):
"""Add all flags in other that aren't in self to self.
Return whether the spec changed.
"""
if other.spec and other.spec._concrete:
for k in self:
if k not in other:
raise UnsatisfiableCompilerFlagSpecError(
self[k], '<absent>')
changed = False
for k in other:
if k in self and not set(self[k]) <= set(other[k]):
raise UnsatisfiableCompilerFlagSpecError(
' '.join(f for f in self[k]),
' '.join(f for f in other[k]))
elif k not in self:
self[k] = other[k]
changed = True
return changed
@staticmethod
def valid_compiler_flags():
return _valid_compiler_flags
def copy(self):
clone = FlagMap(None)
for name, value in self.items():
clone[name] = value
return clone
def _cmp_iter(self):
for k, v in sorted(self.items()):
yield k
def flags():
for flag in v:
yield flag
yield flags
def __str__(self):
sorted_keys = [k for k in sorted(self.keys()) if self[k] != []]
cond_symbol = ' ' if len(sorted_keys) > 0 else ''
return cond_symbol + ' '.join(
str(key) + '=\"' + ' '.join(
str(f) for f in self[key]) + '\"'
for key in sorted_keys) + cond_symbol
def _sort_by_dep_types(dspec):
# Use negation since False < True for sorting
return tuple(t not in dspec.deptypes for t in ("link", "run", "build", "test"))
#: Enum for edge directions
EdgeDirection = lang.enum(parent=0, child=1)
@lang.lazy_lexicographic_ordering
class _EdgeMap(Mapping):
"""Represent a collection of edges (DependencySpec objects) in the DAG.
Objects of this class are used in Specs to track edges that are
outgoing towards direct dependencies, or edges that are incoming
from direct dependents.
Edges are stored in a dictionary and keyed by package name.
"""
def __init__(self, store_by=EdgeDirection.child):
# Sanitize input arguments
msg = 'unexpected value for "store_by" argument'
assert store_by in (EdgeDirection.child, EdgeDirection.parent), msg
#: This dictionary maps a package name to a list of edges
#: i.e. to a list of DependencySpec objects
self.edges = {}
self.store_by_child = (store_by == EdgeDirection.child)
def __getitem__(self, key):
return self.edges[key]
def __iter__(self):
return iter(self.edges)
def __len__(self):
return len(self.edges)
def add(self, edge):
"""Adds a new edge to this object.
Args:
edge (DependencySpec): edge to be added
"""
key = edge.spec.name if self.store_by_child else edge.parent.name
current_list = self.edges.setdefault(key, [])
current_list.append(edge)
current_list.sort(key=_sort_by_dep_types)
def __str__(self):
return "{deps: %s}" % ', '.join(str(d) for d in sorted(self.values()))
def _cmp_iter(self):
for item in sorted(itertools.chain.from_iterable(self.edges.values())):
yield item
def copy(self):
"""Copies this object and returns a clone"""
clone = type(self)()
clone.store_by_child = self.store_by_child
# Copy everything from this dict into it.
for dspec in itertools.chain.from_iterable(self.values()):
clone.add(dspec.copy())
return clone
def select(self, parent=None, child=None, deptypes=dp.all_deptypes):
"""Select a list of edges and return them.
If an edge:
- Has *any* of the dependency types passed as argument,
- Matches the parent and/or child name, if passed
then it is selected.
The deptypes argument needs to be canonical, since the method won't
convert it for performance reason.
Args:
parent (str): name of the parent package
child (str): name of the child package
deptypes (tuple): allowed dependency types in canonical form
Returns:
List of DependencySpec objects
"""
if not deptypes:
return []
# Start from all the edges we store
selected = (d for d in itertools.chain.from_iterable(self.values()))
# Filter by parent name
if parent:
selected = (d for d in selected if d.parent.name == parent)
# Filter by child name
if child:
selected = (d for d in selected if d.spec.name == child)
# Filter by allowed dependency types
if deptypes:
selected = (
dep for dep in selected
if not dep.deptypes or
any(d in deptypes for d in dep.deptypes)
)
return list(selected)
def clear(self):
self.edges.clear()
def _command_default_handler(descriptor, spec, cls):
"""Default handler when looking for the 'command' attribute.
Tries to search for ``spec.name`` in the ``spec.prefix.bin`` directory.
Parameters:
descriptor (ForwardQueryToPackage): descriptor that triggered the call
spec (Spec): spec that is being queried
cls (type(spec)): type of spec, to match the signature of the
descriptor ``__get__`` method
Returns:
Executable: An executable of the command
Raises:
RuntimeError: If the command is not found
"""
path = os.path.join(spec.prefix.bin, spec.name)
if fs.is_exe(path):
return spack.util.executable.Executable(path)
else:
msg = 'Unable to locate {0} command in {1}'
raise RuntimeError(msg.format(spec.name, spec.prefix.bin))
def _headers_default_handler(descriptor, spec, cls):
"""Default handler when looking for the 'headers' attribute.
Tries to search for ``*.h`` files recursively starting from
``spec.prefix.include``.
Parameters:
descriptor (ForwardQueryToPackage): descriptor that triggered the call
spec (Spec): spec that is being queried
cls (type(spec)): type of spec, to match the signature of the
descriptor ``__get__`` method
Returns:
HeaderList: The headers in ``prefix.include``
Raises:
NoHeadersError: If no headers are found
"""
headers = fs.find_headers('*', root=spec.prefix.include, recursive=True)
if headers:
return headers
else:
msg = 'Unable to locate {0} headers in {1}'
raise spack.error.NoHeadersError(
msg.format(spec.name, spec.prefix.include))
def _libs_default_handler(descriptor, spec, cls):
"""Default handler when looking for the 'libs' attribute.
Tries to search for ``lib{spec.name}`` recursively starting from
``spec.prefix``. If ``spec.name`` starts with ``lib``, searches for
``{spec.name}`` instead.
Parameters:
descriptor (ForwardQueryToPackage): descriptor that triggered the call
spec (Spec): spec that is being queried
cls (type(spec)): type of spec, to match the signature of the
descriptor ``__get__`` method
Returns:
LibraryList: The libraries found
Raises:
NoLibrariesError: If no libraries are found
"""
# Variable 'name' is passed to function 'find_libraries', which supports
# glob characters. For example, we have a package with a name 'abc-abc'.
# Now, we don't know if the original name of the package is 'abc_abc'
# (and it generates a library 'libabc_abc.so') or 'abc-abc' (and it
# generates a library 'libabc-abc.so'). So, we tell the function
# 'find_libraries' to give us anything that matches 'libabc?abc' and it
# gives us either 'libabc-abc.so' or 'libabc_abc.so' (or an error)
# depending on which one exists (there is a possibility, of course, to
# get something like 'libabcXabc.so, but for now we consider this
# unlikely).
name = spec.name.replace('-', '?')
# Avoid double 'lib' for packages whose names already start with lib
if not name.startswith('lib'):
name = 'lib' + name
# If '+shared' search only for shared library; if '~shared' search only for
# static library; otherwise, first search for shared and then for static.
search_shared = [True] if ('+shared' in spec) else \
([False] if ('~shared' in spec) else [True, False])
for shared in search_shared:
libs = fs.find_libraries(
name, spec.prefix, shared=shared, recursive=True)
if libs:
return libs
msg = 'Unable to recursively locate {0} libraries in {1}'
raise spack.error.NoLibrariesError(msg.format(spec.name, spec.prefix))
class ForwardQueryToPackage(object):
"""Descriptor used to forward queries from Spec to Package"""
def __init__(self, attribute_name, default_handler=None):
"""Create a new descriptor.
Parameters: