Adding support for ppc64le
amitsadaphule opened this issue · comments
ENV:
arch=ppc64le
os=RHEL7.6
As mentioned in the issue prestodb/presto#14549, in order to get the tests for presto-function-namespace-managers, presto-mysql, presto-verifier, presto-benchmark-runner, presto-raptor to PASS on RHEL 7.6 ppc64le, I needed to add mysql-Linux-ppc64le.tar.gz containing compatible binaries.
So, in order to get that fixed at the source, I need to add the repack script for RHEL/CentOS 7.6 ppc64le. The problem that I see currently is, mysql does not have wide support on ppc64le. On the other hand, mariadb, which is a drop in replacement for mysql, has better support and availability on power. I'll just need to make minor changes (surrounded by if power checks) in some parts of the code (e.g. the initialization args, where --initialize-insecure option is not supported by mariadb).
Will it be an acceptable solution to use mariadb for power?
@gerrith3 FYI.
I think that's reasonable, since there is no mysql alternative.
Thanks @rschlussel for the feedback! I'll work towards that.
@rschlussel I was able to get mariadb 10.2.32 integrated with few code changes by extracting rpm and repacking select files as mysql-Linux-ppc64le.tar.gz. I'm able to get all the tests for 4 dependent packages, namely, presto-verifier, presto-benchmark-runner, presto-raptor, presto-mysql to PASS. However, 2 out of 35 tests for the fifth package presto-function-namespace-managers are failing. Here is the log:
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running TestSuite
2020-06-26T09:46:41.273-0500 INFO Starting MySQL server in /tmp/testing-mysql-server2538724273711170342
2020-06-26T09:46:44.814-0500 INFO mysqld started on port 40659. Waiting up to 10.00s for startup to finish.
2020-06-26 14:46:44 70367358086928 [Note] /tmp/testing-mysql-server2538724273711170342/bin/mysqld (mysqld 10.2.32-MariaDB) starting as process 2485 ...
2020-06-26T09:46:45.143-0500 INFO mysqld startup finished
2020-06-26T09:46:45.152-0500 INFO MySQL server ready: jdbc:mysql://localhost:40659/?user=testuser&password=testpass&useSSL=false
2020-06-26T09:46:45.328-0500 INFO PROPERTY DEFAULT RUNTIME DESCRIPTION
2020-06-26T09:46:45.329-0500 INFO function-cache-expiration 5.00m 0.00s
2020-06-26T09:46:45.330-0500 INFO function-instance-cache-expiration 8.00h 0.00s
2020-06-26T09:46:45.331-0500 INFO database-url ---- jdbc:mysql://localhost:40659/presto?user=testuser&password=testpass&useSSL=false
2020-06-26T09:46:45.332-0500 INFO function-namespaces-table-name function_namespaces function_namespaces
2020-06-26T09:46:45.333-0500 INFO functions-table-name sql_functions sql_functions
2020-06-26T09:46:48.293-0500 INFO Shutting down mysqld. Waiting up to 10.00s for shutdown to finish.
[ERROR] Tests run: 35, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 8.352 s <<< FAILURE! - in TestSuite
[ERROR] testAlterFunctionAmbiguous(com.facebook.presto.functionNamespace.mysql.TestMySqlFunctionNamespaceManager) Time elapsed: 0.093 s <<< FAILURE!
com.facebook.presto.spi.PrestoException: Function 'unittest.memory.power_tower' has multiple signatures: unittest.memory.power_tower(integer):integer; unittest.memory.power_tower(double):double. Please specify parameter types.
at com.facebook.presto.functionNamespace.mysql.MySqlFunctionNamespaceManager.checkUnique(MySqlFunctionNamespaceManager.java:277)
at com.facebook.presto.functionNamespace.mysql.MySqlFunctionNamespaceManager.lambda$alterFunction$1(MySqlFunctionNamespaceManager.java:188)
at org.jdbi.v3.core.Handle.lambda$useTransaction$1(Handle.java:406)
at org.jdbi.v3.core.transaction.LocalTransactionHandler.inTransaction(LocalTransactionHandler.java:135)
at org.jdbi.v3.core.Handle.useTransaction(Handle.java:405)
at org.jdbi.v3.core.Jdbi.lambda$useTransaction$6(Jdbi.java:368)
at org.jdbi.v3.core.Jdbi.lambda$useHandle$4(Jdbi.java:333)
at org.jdbi.v3.core.Jdbi.withHandle(Jdbi.java:318)
at org.jdbi.v3.core.Jdbi.useHandle(Jdbi.java:332)
at org.jdbi.v3.core.Jdbi.useTransaction(Jdbi.java:368)
at com.facebook.presto.functionNamespace.mysql.MySqlFunctionNamespaceManager.alterFunction(MySqlFunctionNamespaceManager.java:184)
at com.facebook.presto.functionNamespace.mysql.TestMySqlFunctionNamespaceManager.alterFunction(TestMySqlFunctionNamespaceManager.java:422)
at com.facebook.presto.functionNamespace.mysql.TestMySqlFunctionNamespaceManager.testAlterFunctionAmbiguous(TestMySqlFunctionNamespaceManager.java:316)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:104)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:645)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:851)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1177)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:129)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:112)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[ERROR] testDropFunctionAmbiguous(com.facebook.presto.functionNamespace.mysql.TestMySqlFunctionNamespaceManager) Time elapsed: 0.049 s <<< FAILURE!
com.facebook.presto.spi.PrestoException: Function 'unittest.memory.power_tower' has multiple signatures: unittest.memory.power_tower(integer):integer; unittest.memory.power_tower(double):double. Please specify parameter types.
at com.facebook.presto.functionNamespace.mysql.MySqlFunctionNamespaceManager.checkUnique(MySqlFunctionNamespaceManager.java:277)
at com.facebook.presto.functionNamespace.mysql.MySqlFunctionNamespaceManager.lambda$dropFunction$2(MySqlFunctionNamespaceManager.java:218)
at org.jdbi.v3.core.Handle.lambda$useTransaction$1(Handle.java:406)
at org.jdbi.v3.core.transaction.LocalTransactionHandler.inTransaction(LocalTransactionHandler.java:135)
at org.jdbi.v3.core.Handle.useTransaction(Handle.java:405)
at org.jdbi.v3.core.Jdbi.lambda$useTransaction$6(Jdbi.java:368)
at org.jdbi.v3.core.Jdbi.lambda$useHandle$4(Jdbi.java:333)
at org.jdbi.v3.core.Jdbi.withHandle(Jdbi.java:318)
at org.jdbi.v3.core.Jdbi.useHandle(Jdbi.java:332)
at org.jdbi.v3.core.Jdbi.useTransaction(Jdbi.java:368)
at com.facebook.presto.functionNamespace.mysql.MySqlFunctionNamespaceManager.dropFunction(MySqlFunctionNamespaceManager.java:214)
at com.facebook.presto.functionNamespace.mysql.TestMySqlFunctionNamespaceManager.dropFunction(TestMySqlFunctionNamespaceManager.java:427)
at com.facebook.presto.functionNamespace.mysql.TestMySqlFunctionNamespaceManager.testDropFunctionAmbiguous(TestMySqlFunctionNamespaceManager.java:364)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:104)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:645)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:851)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1177)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:129)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:112)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[INFO]
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] TestMySqlFunctionNamespaceManager.testAlterFunctionAmbiguous:316->alterFunction:422 » Presto
[ERROR] TestMySqlFunctionNamespaceManager.testDropFunctionAmbiguous:364->dropFunction:427 » Presto
[INFO]
[ERROR] Tests run: 35, Failures: 2, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 22.114 s
[INFO] Finished at: 2020-06-26T14:46:49Z
[INFO] ------------------------------------------------------------------------
I checked TestMySqlFunctionNamespaceManager.java code for the functions testAlterFunctionAmbiguous, testDropFunctionAmbiguous. They seems to be expecting the exception there. Not sure what's going wrong there. Pasting the relevant code snippets:
@Test(expectedExceptions = PrestoException.class, expectedExceptionsMessageRegExp = "Function 'unittest\\.memory\\.power_tower' has multiple signatures: unittest\\.memory\\.power_tower\\(double\\):double; unittest\\.memory\\.power_tower\\(integer\\):integer\\. Please specify parameter types\\.")
public void testAlterFunctionAmbiguous()
{
createFunction(FUNCTION_POWER_TOWER_DOUBLE, false);
createFunction(FUNCTION_POWER_TOWER_INT, false);
alterFunction(POWER_TOWER, Optional.empty(), new AlterRoutineCharacteristics(Optional.of(RETURNS_NULL_ON_NULL_INPUT)));
}
...
@Test(expectedExceptions = PrestoException.class, expectedExceptionsMessageRegExp = "Function 'unittest\\.memory\\.power_tower' has multiple signatures: unittest\\.memory\\.power_tower\\(double\\):double; unittest\\.memory\\.power_tower\\(integer\\):integer\\. Please specify parameter types\\.")
public void testDropFunctionAmbiguous()
{
createFunction(FUNCTION_POWER_TOWER_DOUBLE, false);
createFunction(FUNCTION_POWER_TOWER_INT, false);
dropFunction(POWER_TOWER, Optional.empty(), false);
}
I don't see any failures if I use files copied from local installation of mysql 5.7.22 instead of mariadb 10.2.32 though. Any pointers?
It looks like the error message is ordering the signatures differently (integer then double vs. double then integer). You should be able to change regex to expect either one by replacing both instances with something like (\\(double\\):double | \\(integer\\):integer\\)
@rschlussel I raised the PR (need to fix the travis job yet). But my question is related to the CLA that I got routed to when I clicked on the EASYCLA link. I was routed to LF-Engineering CLA instead of https://code.facebook.com/cla which is mentioned in CONTRIBUTING.md. Are these two CLAs equivalent or is there no relation between the two? I'm asking since I have the approval for signing facebook CLA, but I tried signing the one that I got routed from my PR which triggered a request for a corporate CLA account towards the program director.
Just an update, I've fixed the travis failure.
It looks like the error message is ordering the signatures differently (integer then double vs. double then integer). You should be able to change regex to expect either one by replacing both instances with something like
(\\(double\\):double | \\(integer\\):integer\\)
@rschlussel could you please review this PR: prestodb/presto#14767
@rschlussel I raised the PR (need to fix the travis job yet). But my question is related to the CLA that I got routed to when I clicked on the EASYCLA link. I was routed to LF-Engineering CLA instead of https://code.facebook.com/cla which is mentioned in CONTRIBUTING.md. Are these two CLAs equivalent or is there no relation between the two? I'm asking since I have the approval for signing facebook CLA, but I tried signing the one that I got routed from my PR which triggered a request for a corporate CLA account towards the program director.
thanks. Linux foundation is the right one. We must have forgotten to update contributing.md on some of the prestodb repos, once we be came part of the Linux Foundation.