Skip to content

Commit c0204f7

Browse files
committed
Update LLVM to clang-r487747b.
Includes a bunch of test `#include` fixes for the reason explained in the changelog. Also updates the ASan test. The ASan failure message has been rephrased so the old test no longer matched the death message correctly. As best as I can tell the test we have is a copy of the upstream test with all the supporting functions from the various sanitizer test support headers copied into it. I've replaced the guts of the test and copied the new required supporting functions. This also disables the TSan smoke test. It hangs indefinitely, and we don't have timeout support (`timeout` is not available on old devices). The test was already an xfail, so we can just disable it for now. The test will still be built, so we can be sure that it won't suffer build- time regressions. A few libc++ tests were disabled because the tests (either the failing asserts or whole tests) were removed upstream. Bug: android/ndk#1298 Bug: android/ndk#1677 Bug: android/ndk#1834 Bug: android/ndk#1860 Bug: android/ndk#1862 Test: checkbuild.py && run_tests.py Change-Id: Id467537487908cad99099bfeb2cfacf4787d875c
1 parent b131baf commit c0204f7

File tree

12 files changed

+214
-45
lines changed

12 files changed

+214
-45
lines changed

docs/changelogs/Changelog-r26.md

+24
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,25 @@ directly, see the [build system maintainers guide].
2020

2121
## Changes
2222

23+
* Updated LLVM to clang-r487747b, based on LLVM 17 development.
24+
* Clang now treats `-Wimplicit-function-declaration` as an error rather than a
25+
warning in C11 and newer. Clang's default C standard is 17, so this is a
26+
change in default behavior compared to older versions of Clang, but is the
27+
behavior defined by C99.
28+
29+
If you encounter these errors when upgrading, you most likely forgot an
30+
`#include`. If you cannot (or do not want to) fix those issues, you can
31+
revert to the prior behavior with
32+
`-Wno-error=implicit-function-declaration`.
33+
34+
C++ users are unaffected. This has never been allowed in C++.
35+
36+
See https://reviews.llvm.org/D122983 for more details.
37+
* [Issue 1298]: Fixed seccomp error with ASan on x86_64 devices.
38+
* [Issue 1677]: Fixed Clang crash in optimizer.
39+
* [Issue 1834]: Fixed Clang crash during SVE conversions.
40+
* [Issue 1860]: Fixed miscompilation affecting armv7.
41+
* [Issue 1862]: Fixed Clang crash for aarch64 with `-Os`.
2342
* Version scripts that name public symbols that are not present in the library
2443
will now emit an error by default for ndk-build and the CMake toolchain file.
2544
Build failures caused by this error are likely a bug in your library or a
@@ -36,6 +55,11 @@ directly, see the [build system maintainers guide].
3655

3756
[Issue 837]: https://github.com/android/ndk/issues/837
3857
[Issue 1803]: https://github.com/android/ndk/issues/1803
58+
[Issue 1298]: https://github.com/android/ndk/issues/1298
59+
[Issue 1677]: https://github.com/android/ndk/issues/1677
60+
[Issue 1834]: https://github.com/android/ndk/issues/1834
61+
[Issue 1860]: https://github.com/android/ndk/issues/1860
62+
[Issue 1862]: https://github.com/android/ndk/issues/1862
3963

4064
## Known Issues
4165

ndk/checkbuild.py

+43-20
Original file line numberDiff line numberDiff line change
@@ -469,7 +469,7 @@ def install(self) -> None:
469469
(bin_dir / f"lld{bin_ext}").unlink()
470470
(bin_dir / f"lld-link{bin_ext}").unlink()
471471

472-
install_clanglib = install_path / "lib64/clang"
472+
install_clanglib = install_path / "lib/clang"
473473
linux_prebuilt_path = ClangToolchain.path_for_host(Host.Linux)
474474

475475
# Remove unused python scripts. They are not installed for Windows.
@@ -487,25 +487,18 @@ def install(self) -> None:
487487
for pyfile in python_bin_dir.glob(file_pattern):
488488
pyfile.unlink()
489489

490-
# Remove lldb-argdumper in site-packages. libc++ is not available there.
491-
# People should use bin/lldb-argdumper instead.
492-
for pylib in (install_path / "lib").glob("python*"):
493-
(pylib / f"site-packages/lldb/lldb-argdumper{bin_ext}").unlink()
494-
495490
if self.host != Host.Linux:
496-
# We don't build target binaries as part of the Darwin or Windows
497-
# build. These toolchains need to get these from the Linux
498-
# prebuilts.
491+
# We don't build target binaries as part of the Darwin or Windows build.
492+
# These toolchains need to get these from the Linux prebuilts.
499493
#
500-
# The headers and libraries we care about are all in lib64/clang
501-
# for both toolchains, and those two are intended to be identical
502-
# between each host, so we can just replace them with the one from
503-
# the Linux toolchain.
494+
# The headers and libraries we care about are all in lib/clang for both
495+
# toolchains, and those two are intended to be identical between each host,
496+
# so we can just replace them with the one from the Linux toolchain.
504497
shutil.rmtree(install_clanglib)
505-
shutil.copytree(linux_prebuilt_path / "lib64/clang", install_clanglib)
498+
shutil.copytree(linux_prebuilt_path / "lib/clang", install_clanglib)
506499

507-
# The Clang prebuilts have the platform toolchain libraries in
508-
# lib64/clang. The libraries we want are in runtimes_ndk_cxx.
500+
# The Clang prebuilts have the platform toolchain libraries in lib/clang. The
501+
# libraries we want are in runtimes_ndk_cxx.
509502
ndk_runtimes = linux_prebuilt_path / "runtimes_ndk_cxx"
510503
for version_dir in install_clanglib.iterdir():
511504
dst_lib_dir = version_dir / "lib/linux"
@@ -541,15 +534,15 @@ def install(self) -> None:
541534
# For some reason the LLVM install includes CMake modules that expose
542535
# its internal APIs. We want to purge these so apps don't accidentally
543536
# depend on them. See http://b/142327416 for more info.
544-
shutil.rmtree(install_path / "lib64/cmake")
537+
shutil.rmtree(install_path / "lib/cmake")
545538

546539
# Remove libc++.a and libc++abi.a on Darwin. Now that these files are
547540
# universal binaries, they break notarization. Maybe it is possible to
548541
# fix notarization by using ditto to preserve APFS extended attributes.
549542
# See https://developer.apple.com/forums/thread/126038.
550543
if self.host == Host.Darwin:
551-
(install_path / "lib64/libc++.a").unlink()
552-
(install_path / "lib64/libc++abi.a").unlink()
544+
(install_path / "lib/libc++.a").unlink()
545+
(install_path / "lib/libc++abi.a").unlink()
553546

554547
# Strip some large binaries and libraries. This is awkward, hand-crafted
555548
# logic to select most of the biggest offenders, but could be
@@ -578,6 +571,36 @@ def install(self) -> None:
578571
):
579572
subprocess.check_call([str(strip_cmd), "--strip-unneeded", str(file)])
580573

574+
for lib in (install_path / "lib").iterdir():
575+
broken_symlinks = {
576+
"libc++abi.so.1.0",
577+
"libc++abi.so",
578+
"libc++.so.1.0",
579+
}
580+
581+
if lib.name in broken_symlinks:
582+
self._check_and_remove_dangling_symlink(lib)
583+
584+
def _check_and_remove_dangling_symlink(self, path: Path) -> None:
585+
"""Removes an expected dangling symlink, or raises an error.
586+
587+
The latest LLVM prebuilts have some dangling symlinks. It's a bug on the LLVM
588+
build side, but rather than wait for a respin we just clean up the problems
589+
here. This will raise an error whenever we upgrade to a new toolchain that
590+
doesn't have these problems, so we'll know when to remove the workaround.
591+
"""
592+
if not path.is_symlink():
593+
raise RuntimeError(
594+
f"Expected {path} to be a symlink. Update or remove this workaround."
595+
)
596+
if (dest := path.readlink()).exists():
597+
raise RuntimeError(
598+
f"Expected {path} to be a dangling symlink, but {dest} exists. Update "
599+
"or remove this workaround."
600+
)
601+
602+
path.unlink()
603+
581604

582605
def versioned_so(host: Host, lib: str, version: str) -> str:
583606
"""Returns the formatted versioned library for the given host.
@@ -665,7 +688,7 @@ def env(self) -> Dict[str, str]:
665688

666689
@property
667690
def _libcxx_dir(self) -> Path:
668-
return self.get_dep("clang").get_build_host_install() / "lib64"
691+
return self.get_dep("clang").get_build_host_install() / "lib"
669692

670693
@property
671694
def _libcxx(self) -> List[Path]:

ndk/toolchains.py

+1-2
Original file line numberDiff line numberDiff line change
@@ -21,8 +21,7 @@
2121
from ndk.hosts import Host, get_default_host
2222
import ndk.paths
2323

24-
25-
CLANG_VERSION = "clang-r450784d1"
24+
CLANG_VERSION = "clang-r487747b"
2625

2726

2827
HOST_TRIPLE_MAP = {

tests/device/asan-smoke/jni/asan_oob_test.cc

+109-16
Original file line numberDiff line numberDiff line change
@@ -6,10 +6,20 @@
66
// License. See LICENSE.TXT for details.
77
//
88
//===----------------------------------------------------------------------===//
9+
#include <stdint.h>
910
#include <stdio.h>
11+
#include <stdlib.h>
12+
13+
#include <string>
1014

1115
#include <gtest/gtest.h>
1216

17+
#if __LP64__ || defined(_WIN64)
18+
# define SANITIZER_WORDSIZE 64
19+
#else
20+
# define SANITIZER_WORDSIZE 32
21+
#endif
22+
1323
#define NOINLINE __attribute__((noinline))
1424

1525
typedef uint8_t U1;
@@ -26,6 +36,16 @@ inline void break_optimization(void *arg) {
2636
#endif
2737
}
2838

39+
// This function returns its parameter but in such a way that compiler
40+
// can not prove it.
41+
template<class T>
42+
NOINLINE
43+
static T Ident(T t) {
44+
T ret = t;
45+
break_optimization(&ret);
46+
return ret;
47+
}
48+
2949
NOINLINE void *malloc_fff(size_t size) {
3050
void *res = malloc/**/(size); break_optimization(0); return res;}
3151
NOINLINE void *malloc_eee(size_t size) {
@@ -48,6 +68,14 @@ NOINLINE void asan_write(T *a) {
4868
*a = 0;
4969
}
5070

71+
NOINLINE void asan_write_sized_aligned(uint8_t *p, size_t size) {
72+
EXPECT_EQ(0U, ((uintptr_t)p % size));
73+
if (size == 1) asan_write((uint8_t*)p);
74+
else if (size == 2) asan_write((uint16_t*)p);
75+
else if (size == 4) asan_write((uint32_t*)p);
76+
else if (size == 8) asan_write((uint64_t*)p);
77+
}
78+
5179
template<typename T>
5280
NOINLINE void oob_test(int size, int off) {
5381
char *p = (char*)malloc_aaa(size);
@@ -57,41 +85,106 @@ NOINLINE void oob_test(int size, int off) {
5785
free_aaa(p);
5886
}
5987

88+
static std::string GetLeftOOBMessage(int off) {
89+
char str[100];
90+
sprintf(str, "is located.*%d byte.*before", off);
91+
return str;
92+
}
93+
94+
static std::string GetRightOOBMessage(int off) {
95+
char str[100];
96+
#if !defined(_WIN32)
97+
// FIXME: Fix PR42868 and remove SEGV match.
98+
sprintf(str, "is located.*%d byte.*after|SEGV", off);
99+
#else
100+
// `|` doesn't work in googletest's regexes on Windows,
101+
// see googletest/docs/advanced.md#regular-expression-syntax
102+
// But it's not needed on Windows anyways.
103+
sprintf(str, "is located.*%d byte.*after", off);
104+
#endif
105+
return str;
106+
}
107+
60108
template<typename T>
61109
void OOBTest() {
62-
char expected_str[100];
63110
for (int size = sizeof(T); size < 20; size += 5) {
64-
for (int i = -5; i < 0; i++) {
65-
const char *str =
66-
"is located.*%d byte.*to the left";
67-
sprintf(expected_str, str, abs(i));
68-
EXPECT_DEATH(oob_test<T>(size, i), expected_str);
69-
}
111+
for (int i = -5; i < 0; i++)
112+
EXPECT_DEATH(oob_test<T>(size, i), GetLeftOOBMessage(-i));
70113

71114
for (int i = 0; i < (int)(size - sizeof(T) + 1); i++)
72115
oob_test<T>(size, i);
73116

74117
for (int i = size - sizeof(T) + 1; i <= (int)(size + 2 * sizeof(T)); i++) {
75-
const char *str =
76-
"is located.*%d byte.*to the right";
77-
int off = i >= size ? (i - size) : 0;
78118
// we don't catch unaligned partially OOB accesses.
79119
if (i % sizeof(T)) continue;
80-
sprintf(expected_str, str, off);
81-
EXPECT_DEATH(oob_test<T>(size, i), expected_str);
120+
int off = i >= size ? (i - size) : 0;
121+
EXPECT_DEATH(oob_test<T>(size, i), GetRightOOBMessage(off));
82122
}
83123
}
84124

85-
EXPECT_DEATH(oob_test<T>(kLargeMalloc, -1),
86-
"is located.*1 byte.*to the left");
87-
EXPECT_DEATH(oob_test<T>(kLargeMalloc, kLargeMalloc),
88-
"is located.*0 byte.*to the right");
125+
EXPECT_DEATH(oob_test<T>(kLargeMalloc, -1), GetLeftOOBMessage(1));
126+
EXPECT_DEATH(oob_test<T>(kLargeMalloc, kLargeMalloc), GetRightOOBMessage(0));
89127
}
90128

129+
// TODO(glider): the following tests are EXTREMELY slow on Darwin:
130+
// AddressSanitizer.OOB_char (125503 ms)
131+
// AddressSanitizer.OOB_int (126890 ms)
132+
// AddressSanitizer.OOBRightTest (315605 ms)
133+
// AddressSanitizer.SimpleStackTest (366559 ms)
134+
91135
TEST(AddressSanitizer, OOB_char) {
92136
OOBTest<U1>();
93137
}
94138

95139
TEST(AddressSanitizer, OOB_int) {
96140
OOBTest<U4>();
97141
}
142+
143+
TEST(AddressSanitizer, OOBRightTest) {
144+
size_t max_access_size = SANITIZER_WORDSIZE == 64 ? 8 : 4;
145+
for (size_t access_size = 1; access_size <= max_access_size;
146+
access_size *= 2) {
147+
for (size_t alloc_size = 1; alloc_size <= 8; alloc_size++) {
148+
for (size_t offset = 0; offset <= 8; offset += access_size) {
149+
void *p = malloc(alloc_size);
150+
// allocated: [p, p + alloc_size)
151+
// accessed: [p + offset, p + offset + access_size)
152+
uint8_t *addr = (uint8_t*)p + offset;
153+
if (offset + access_size <= alloc_size) {
154+
asan_write_sized_aligned(addr, access_size);
155+
} else {
156+
int outside_bytes = offset > alloc_size ? (offset - alloc_size) : 0;
157+
EXPECT_DEATH(asan_write_sized_aligned(addr, access_size),
158+
GetRightOOBMessage(outside_bytes));
159+
}
160+
free(p);
161+
}
162+
}
163+
}
164+
}
165+
166+
TEST(AddressSanitizer, LargeOOBRightTest) {
167+
size_t large_power_of_two = 1 << 19;
168+
for (size_t i = 16; i <= 256; i *= 2) {
169+
size_t size = large_power_of_two - i;
170+
char *p = Ident(new char[size]);
171+
EXPECT_DEATH(p[size] = 0, GetRightOOBMessage(0));
172+
delete [] p;
173+
}
174+
}
175+
176+
TEST(AddressSanitizer, DISABLED_DemoOOBLeftLow) {
177+
oob_test<U1>(10, -1);
178+
}
179+
180+
TEST(AddressSanitizer, DISABLED_DemoOOBLeftHigh) {
181+
oob_test<U1>(kLargeMalloc, -1);
182+
}
183+
184+
TEST(AddressSanitizer, DISABLED_DemoOOBRightLow) {
185+
oob_test<U1>(10, 10);
186+
}
187+
188+
TEST(AddressSanitizer, DISABLED_DemoOOBRightHigh) {
189+
oob_test<U1>(kLargeMalloc, kLargeMalloc);
190+
}

tests/device/clone/jni/clone.c

+3-1
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,9 @@
1+
#define _GNU_SOURCE
2+
13
#include <stdio.h>
24
#include <unistd.h>
35
#include <fcntl.h>
4-
#include <linux/sched.h>
6+
#include <sched.h>
57
#include <stdlib.h>
68

79
int v, fd;

tests/device/issue19851-sigsetjmp/jni/issue19851-sigsetjmp.c

+1
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@
1717
#include <stdio.h>
1818
#include <signal.h>
1919
#include <setjmp.h>
20+
#include <unistd.h>
2021

2122
static sigjmp_buf sbuf;
2223

tests/device/openmp/jni/fib.c

+6-3
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,11 @@
1+
#include <math.h>
12
#include <stdio.h>
23
#include <stdlib.h>
3-
#include <math.h>
4-
#include <omp.h>
4+
#include <sys/time.h>
55
#include <unistd.h>
6+
7+
#include <omp.h>
8+
69
#define MAX 33 //41
710
int Fibonacci(int n)
811
{ int x, y;
@@ -31,7 +34,7 @@ int main(int argc, char * argv[])
3134
struct timeval time_start, time_end;
3235
int i = 0;
3336
// openmp related print message
34-
printf("CPU_ONLN= %d\n", sysconf(_SC_NPROCESSORS_ONLN));
37+
printf("CPU_ONLN= %ld\n", sysconf(_SC_NPROCESSORS_ONLN));
3538
printf("Number of CPUs=%d\n", omp_get_num_procs());
3639
printf("Number of max threads=%d\n", omp_get_max_threads());
3740
printf("Number of executing thread=%d\n", omp_get_thread_num());

tests/device/openmp/jni/openmp2.c

+2-2
Original file line numberDiff line numberDiff line change
@@ -6,8 +6,8 @@
66
int main (int argc, char *argv[])
77
{
88
int nthreads, tid;
9-
printf("SC_NPROCESSORS_ONLN: %d\n", sysconf (_SC_NPROCESSORS_ONLN));
10-
printf("SC_NPROCESSORS_CONF: %d\n", sysconf (_SC_NPROCESSORS_CONF));
9+
printf("SC_NPROCESSORS_ONLN: %ld\n", sysconf (_SC_NPROCESSORS_ONLN));
10+
printf("SC_NPROCESSORS_CONF: %ld\n", sysconf (_SC_NPROCESSORS_CONF));
1111
#pragma omp parallel default(shared) private(nthreads, tid)
1212
/* Fork a team of threads giving them their own copies of variables */
1313
{

tests/device/test-cpufeatures/jni/test_arm_idiv.c

+3-1
Original file line numberDiff line numberDiff line change
@@ -13,11 +13,13 @@
1313
* See the License for the specific language governing permissions and
1414
* limitations under the License.
1515
*/
16-
#include <cpu-features.h>
16+
#include <memory.h>
1717
#include <setjmp.h>
1818
#include <stdio.h>
1919
#include <signal.h>
2020

21+
#include <cpu-features.h>
22+
2123
#ifndef __arm__
2224
#error "Only compile this file for an ARM target"
2325
#endif

0 commit comments

Comments
 (0)