Giters

Azure / Azurite

A lightweight server clone of Azure Storage that simulates most of the commands supported by it with minimal dependencies

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Cannot create a blob container from Azure Java client 12.26.0

xdelox opened this issue · comments

commented

Which service(blob, file, queue, table) does this issue concern?

Blob

Which version of the Azurite was used?

3.30

Where do you get Azurite? (npm, DockerHub, NuGet, Visual Studio Code Extension)

DockerHub

What's the Node.js version?

Running Azurite via Docker, so the one shipped with such version of Azurite.

What problem was encountered?

I'm using Azure Blob Storage Java library (see pom.xml snippet below).

        <dependency>
            <groupId>com.azure</groupId>
            <artifactId>azure-storage-blob</artifactId>
            <version>12.26.0</version>
            <scope>test</scope>
        </dependency>

Basically I cannot create a container against Azurite, which returns the following error in the console:

com.azure.storage.blob.models.BlobStorageException: Status code 400, (empty body)

	at java.base/java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:732)
	at com.azure.core.implementation.MethodHandleReflectiveInvoker.invokeWithArguments(MethodHandleReflectiveInvoker.java:39)
	at com.azure.core.implementation.http.rest.ResponseExceptionConstructorCache.invoke(ResponseExceptionConstructorCache.java:53)
	at com.azure.core.implementation.http.rest.RestProxyBase.instantiateUnexpectedException(RestProxyBase.java:408)
	at com.azure.core.implementation.http.rest.AsyncRestProxy.lambda$ensureExpectedStatus$1(AsyncRestProxy.java:137)
	at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:113)
	at reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2400)
	at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:171)
	at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.set(Operators.java:2196)
	at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.onSubscribe(Operators.java:2070)
	at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:96)
	at reactor.core.publisher.MonoJust.subscribe(MonoJust.java:55)
	at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
	at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:157)
	at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:129)
	at reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onNext(FluxHide.java:137)
	at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:129)
	at reactor.core.publisher.FluxHide$SuppressFuseableSubscriber.onNext(FluxHide.java:137)
	at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:79)
	at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1839)
	at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:151)
	at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:129)
	at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1839)
	at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:151)
	at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:79)
	at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1839)
	at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:151)
	at reactor.core.publisher.Operators$MonoInnerProducerBase.complete(Operators.java:2666)
	at reactor.core.publisher.MonoSingle$SingleSubscriber.onComplete(MonoSingle.java:180)
	at reactor.core.publisher.MonoFlatMapMany$FlatMapManyInner.onComplete(MonoFlatMapMany.java:260)
	at reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.onComplete(FluxContextWrite.java:126)
	at reactor.core.publisher.MonoUsing$MonoUsingSubscriber.onNext(MonoUsing.java:232)
	at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:122)
	at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:74)
	at reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2400)
	at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.set(Operators.java:2196)
	at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.onSubscribe(Operators.java:2070)
	at reactor.core.publisher.MonoJust.subscribe(MonoJust.java:55)
	at reactor.core.publisher.Mono.subscribe(Mono.java:4491)
	at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onComplete(FluxSwitchIfEmpty.java:82)
	at reactor.core.publisher.FluxHandle$HandleSubscriber.onComplete(FluxHandle.java:220)
	at reactor.core.publisher.FluxMap$MapConditionalSubscriber.onComplete(FluxMap.java:275)
	at reactor.core.publisher.FluxDoFinally$DoFinallySubscriber.onComplete(FluxDoFinally.java:128)
	at reactor.core.publisher.FluxHandleFuseable$HandleFuseableSubscriber.onNext(FluxHandleFuseable.java:208)
	at reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.onNext(FluxContextWrite.java:107)
	at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1839)
	at reactor.core.publisher.MonoCollectList$MonoCollectListSubscriber.onComplete(MonoCollectList.java:129)
	at reactor.core.publisher.FluxPeek$PeekSubscriber.onComplete(FluxPeek.java:260)
	at reactor.core.publisher.FluxMap$MapSubscriber.onComplete(FluxMap.java:144)
	at reactor.netty.channel.FluxReceive.onInboundComplete(FluxReceive.java:415)
	at reactor.netty.channel.ChannelOperations.onInboundComplete(ChannelOperations.java:439)
	at reactor.netty.channel.ChannelOperations.terminate(ChannelOperations.java:493)
	at reactor.netty.http.client.HttpClientOperations.onInboundNext(HttpClientOperations.java:789)
	at reactor.netty.channel.ChannelOperationsHandler.channelRead(ChannelOperationsHandler.java:114)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at com.azure.core.http.netty.implementation.AzureSdkHandler.channelRead(AzureSdkHandler.java:224)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318)
	at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
	at io.netty.channel.kqueue.AbstractKQueueStreamChannel$KQueueStreamUnsafe.readReady(AbstractKQueueStreamChannel.java:544)
	at io.netty.channel.kqueue.AbstractKQueueChannel$AbstractKQueueUnsafe.readReady(AbstractKQueueChannel.java:387)
	at io.netty.channel.kqueue.KQueueEventLoop.processReady(KQueueEventLoop.java:218)
	at io.netty.channel.kqueue.KQueueEventLoop.run(KQueueEventLoop.java:296)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:833)
	Suppressed: java.lang.Exception: #block terminated with an error
		at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:100)
		at reactor.core.publisher.Mono.block(Mono.java:1742)
		at com.azure.storage.common.implementation.StorageImplUtils.blockWithOptionalTimeout(StorageImplUtils.java:151)
		at com.azure.storage.blob.BlobContainerClient.createIfNotExistsWithResponse(BlobContainerClient.java:374)
		at com.azure.storage.blob.BlobServiceClient.createBlobContainerIfNotExistsWithResponse(BlobServiceClient.java:210)
		at com.azure.storage.blob.BlobServiceClient.createBlobContainerIfNotExists(BlobServiceClient.java:172)
		at org.apache.hop.workflow.actions.movefiles.AzuriteBlobStorageIT.createContainer(AzuriteBlobStorageIT.java:48)
		at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
		at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
		at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
		at java.base/java.lang.reflect.Method.invoke(Method.java:568)
		at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
		at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
		at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
		at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
		at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
		at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
		at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
		at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
		at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
		at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
		at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
		at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
		at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
		at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
		at org.testcontainers.containers.FailureDetectingExternalResource$1.evaluate(FailureDetectingExternalResource.java:29)
		at org.junit.rules.RunRules.evaluate(RunRules.java:20)
		at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
		at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
		at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
		at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
		at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
		at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
		at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
		at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:232)
		at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:55)

Steps to reproduce the issue?

To get the stacktrace above, I'm basically running the following test, based on an Azurite Testcontainer:

public class AzuriteBlobStorageIT {

    private static String CONNECTION_STRING;

    private static final String CONTAINER_NAME = "thisisanewcontainer";
    public static final String AZURITE_DEFAULT_ACCOUNT = "devstoreaccount1";
    public static final String AZURITE_DEFAULT_KEY =
        "Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==";

    private static BlobServiceClient blobServiceClient;

    @ClassRule
    public static GenericContainer azuriteContainer =
        new GenericContainer<>("mcr.microsoft.com/azure-storage/azurite:3.30.0")
            .withExposedPorts(10000)
            .withCommand(
                "azurite-blob", "--blobHost", "0.0.0.0", "--loose", "--debug", "/home/debug.log");

    @Test
    public void createContainer() throws InterruptedException {
        loadAzureProperties();
        blobServiceClient = new BlobServiceClientBuilder().connectionString(CONNECTION_STRING).buildClient();
        blobServiceClient.createBlobContainerIfNotExists(CONTAINER_NAME);
    }

    private static void loadAzureProperties() {
        String host = azuriteContainer.getContainerIpAddress();
        Integer port = azuriteContainer.getMappedPort(10000);

        CONNECTION_STRING =
            String.format(
                "AccountName=%s;AccountKey=%s;DefaultEndpointsProtocol=http;BlobEndpoint=http://%s:%d;",
                AZURITE_DEFAULT_ACCOUNT, AZURITE_DEFAULT_KEY, host, port);

    }


}

Here following you can find the debug log, which seems to highlight some issue with the REST request.

2024-06-04T11:02:20.743Z 	 info: Azurite Blob service is starting on 0.0.0.0:10000
2024-06-04T11:02:20.746Z 	 info: AccountDataStore:init() Refresh accounts from environment variable AZURITE_ACCOUNTS with value undefined
2024-06-04T11:02:20.746Z 	 info: AccountDataStore:init() Fallback to default emulator account devstoreaccount1.
2024-06-04T11:02:20.776Z 	 info: BlobGCManager:start() Starting BlobGCManager. Set status to Initializing.
2024-06-04T11:02:20.776Z 	 info: BlobGCManager:start() Trigger mark and sweep loop. Set status to Running.
2024-06-04T11:02:20.776Z 	 info: BlobGCManager:markSweepLoop() Start next mark and sweep.
2024-06-04T11:02:20.777Z 	 info: BlobGCManager:markSweep() Get all extents.
2024-06-04T11:02:20.779Z 	 info: BlobGCManager:start() BlobGCManager successfully started.
2024-06-04T11:02:20.783Z 	 info: BlobGCManager:markSweep() Got 0 extents.
2024-06-04T11:02:20.784Z 	 info: BlobGCManager:markSweep() Get referred extents.
2024-06-04T11:02:20.785Z 	 info: BlobGCManager:markSweep() Got referred extents, unreferenced extents count is 0.
2024-06-04T11:02:20.785Z 	 info: BlobGCManager:markSweepLoop() Mark and sweep finished, taken 8ms.
2024-06-04T11:02:20.786Z 	 info: BlobGCManager:markSweepLoop() Sleep for 600000ms.
2024-06-04T11:02:20.787Z 	 info: Azurite Blob service successfully listens on http://0.0.0.0:10000
2024-06-04T11:02:21.859Z 5e6d6e3e-9319-42a5-ad2d-d0cbfa819673 info: BlobStorageContextMiddleware: RequestMethod=PUT RequestURL=http://localhost/thisisanewcontainer?restype=container RequestHeaders:{"host":"localhost:52904","date":"Tue, 04 Jun 2024 11:02:21 GMT","authorization":"SharedKey devstoreaccount1:tLrAo+VynGLJhOOn793NuuZ+Cw7VdG3WBG1NQYY0z0Q=","x-ms-version":"2024-05-04","x-ms-client-request-id":"30de8fab-551f-409a-801f-17caeb0b45b0","accept":"application/xml","user-agent":"azsdk-java-azure-storage-blob/12.26.0 (17.0.1; Mac OS X; 14.5)","content-length":"0"} ClientIP=172.17.0.1 Protocol=http HTTPVersion=1.1
2024-06-04T11:02:21.860Z 5e6d6e3e-9319-42a5-ad2d-d0cbfa819673 info: BlobStorageContextMiddleware: Account=thisisanewcontainer Container=undefined Blob=
2024-06-04T11:02:21.861Z 5e6d6e3e-9319-42a5-ad2d-d0cbfa819673 verbose: DispatchMiddleware: Dispatching request...
2024-06-04T11:02:21.866Z 5e6d6e3e-9319-42a5-ad2d-d0cbfa819673 error: DispatchMiddleware: Incoming URL doesn't match any of swagger defined request patterns.
2024-06-04T11:02:21.868Z 5e6d6e3e-9319-42a5-ad2d-d0cbfa819673 error: ErrorMiddleware: Received a MiddlewareError, fill error information to HTTP response
2024-06-04T11:02:21.868Z 5e6d6e3e-9319-42a5-ad2d-d0cbfa819673 error: ErrorMiddleware: ErrorName=UnsupportedRequestError ErrorMessage=Incoming URL doesn't match any of swagger defined request patterns.  ErrorHTTPStatusCode=400 ErrorHTTPStatusMessage=undefined ErrorHTTPHeaders=undefined ErrorHTTPBody=undefined ErrorStack="UnsupportedRequestError: Incoming URL doesn't match any of swagger defined request patterns.\n    at dispatchMiddleware (/opt/azurite/dist/src/blob/generated/middleware/dispatch.middleware.js:41:30)\n    at /opt/azurite/dist/src/blob/generated/ExpressMiddlewareFactory.js:50:47\n    at Layer.handle [as handle_request] (/opt/azurite/node_modules/express/lib/router/layer.js:95:5)\n    at trim_prefix (/opt/azurite/node_modules/express/lib/router/index.js:328:13)\n    at /opt/azurite/node_modules/express/lib/router/index.js:286:9\n    at Function.process_params (/opt/azurite/node_modules/express/lib/router/index.js:346:12)\n    at next (/opt/azurite/node_modules/express/lib/router/index.js:280:10)\n    at blobStorageContextMiddleware (/opt/azurite/dist/src/blob/middlewares/blobStorageContext.middleware.js:137:5)\n    at /opt/azurite/dist/src/blob/middlewares/blobStorageContext.middleware.js:15:16\n    at Layer.handle [as handle_request] (/opt/azurite/node_modules/express/lib/router/layer.js:95:5)"
2024-06-04T11:02:21.868Z 5e6d6e3e-9319-42a5-ad2d-d0cbfa819673 error: ErrorMiddleware: Set HTTP code: 400
2024-06-04T11:02:21.869Z 5e6d6e3e-9319-42a5-ad2d-d0cbfa819673 error: ErrorMiddleware: Set HTTP body: undefined
2024-06-04T11:02:21.870Z 5e6d6e3e-9319-42a5-ad2d-d0cbfa819673 info: EndMiddleware: End response. TotalTimeInMS=10 StatusCode=400 StatusMessage=undefined Headers={"server":"Azurite-Blob/3.30.0"}

Have you found a mitigation/solution?

Not yet, but I tried to analyse the debug log when trying to do the same with Ms Azure Storage Explorer (of course it works) and there are some differences with headers for sure, but I can't say yet whether it's an offending header or not.

commented

@xdelox

The request send to Azurite is not valid, this is a client issue.

From the Azurite debug log, you can see the request Uri is "http://localhost/thisisanewcontainer?restype=container", the storage account name is missed from the Uri, so this is a client issue that request send from client is not valid. In this case, a valid Uri in Debug log should look like "http://localhost/devstoreaccount1/thisisanewcontainer?restype=container".

commented

@xdelox

I will close this issue as it's a client issue and not get further responds from you.
Feel free to contact us again if need any further assistance on Azurite.

commented