SolrCloud avec Docker Compose

Cet article sur la mise en place de SolrCloud fait écho à celui sur le cluster Cassandra avec Docker. Pour comprendre certains points abordés ci-dessous, il est nécessaire de connaitre la terminologie SolrCloud.

Premiers essais avec Docker

Comme précédemment, la première étape se résume à valider les images et la communication entre les conteneurs du cluster du moteur d’indexation via la ligne de commande docker. Comme souvent, l’équipe Docker propose une image officielle pour Solr avec un mode distribué qui peut être lancé de cette façon :

$ docker run --name myproject-zookeeper -d -p 2181:2181 -p 2888:2888 -p 3888:3888 jplock/zookeeper

# démarrage de 4 instances Solr
$ for i in `seq 1 4`;
do
  docker run --name myproject-solr-$i --link myproject-zookeeper:ZK -d -p $(( 8982 + $i )):8983 solr bash -c '/opt/solr/bin/solr start -f -z $ZK_PORT_2181_TCP_ADDR:$ZK_PORT_2181_TCP_PORT';
done

Cette image contient également les outils pour créer une Collection. Pour cela, il faut l’exécuter directement sur l’un des conteneurs du cluster. Voici la commande pour créer une Collection découpée en 2 Shards eux-mêmes répliqués 2 fois :

$ docker exec -i -t myproject-solr-1 bin/solr create_collection -c myproject -shards 2 -replicationFactor 2 -p 8983
Connecting to ZooKeeper at 172.17.0.5:2181 ...
Uploading /opt/solr/server/solr/configsets/data_driven_schema_configs/conf for config myproject to ZooKeeper at 172.17.0.5:2181

Creating new collection 'myproject' using command:
http://localhost:8983/solr/admin/collections?action=CREATE&name=myproject&numShards=2&replicationFactor=2&maxShardsPerNode=1&collection.configName=myproject

{
  "responseHeader":{
    "status":0,
    "QTime":8746},
  "success":{"":{
      "responseHeader":{
        "status":0,
        "QTime":8478},
      "core":"myproject_shard1_replica2"}}}

En se rendant sur la console d’administration de SolrCloud (http://dockermachine:8983/solr/#/~cloud), il est possible de constater la création de la collection distribuée sur les 4 instances :

Il ne reste plus qu’à lancer l’indexation de documents toujours en utilisant les outils mis à disposition par Solr directement dans l’un des conteneurs du cluster :

$ docker exec -it --user=solr myproject-solr-1 bin/post -c myproject example/exampledocs/manufacturers.xml
java -classpath /opt/solr/dist/solr-core-5.3.1.jar -Dauto=yes -Dc=myproject -Ddata=files org.apache.solr.util.SimplePostTool example/exampledocs/manufacturers.xml
SimplePostTool version 5.0.0
Posting files to [base] url http://localhost:8983/solr/myproject/update...
Entering auto mode. File endings considered are xml,json,csv,pdf,doc,docx,ppt,pptx,xls,xlsx,odt,odp,ods,ott,otp,ots,rtf,htm,html,txt,log
POSTing file manufacturers.xml (application/xml) to [base]
1 files indexed.
COMMITting Solr index changes to http://localhost:8983/solr/myproject/update...
Time spent: 0:00:01.150

Puis de vérifier la réplication sur chacun des noeuds :

for i in `seq 1 4`;
do
	curl "http://dockermachine:$(( 8982 + $i ))/solr/myproject/select?q=*%3A*&wt=json"
done
{"responseHeader":{"status":0,"QTime":60,"params":{"q":"*:*","wt":"json"}},"response":{"numFound":11,"start":0,"maxScore":1.0,"docs":[{"id":"adata","compName_s":"A-Data Technology","address_s":"46221 Landing Parkway Fremont, CA 94538","_version_":1518135858246451200},{"id":"asus","compName_s":"ASUS Computer","address_s":"800 Corporate Way Fremont, CA 94539","_version_":1518135858502303744},{"id":"belkin","compName_s":"Belkin","address_s":"12045 E. Waterfront Drive Playa Vista, CA 90094","_version_":1518135858504400896},{"id":"maxtor","compName_s":"Maxtor Corporation","address_s":"920 Disc Drive Scotts Valley, CA 95066","_version_":1518135858511740928},{"id":"apple","compName_s":"Apple","address_s":"1 Infinite Way, Cupertino CA","_version_":1518135858281054208},{"id":"ati","compName_s":"ATI Technologies","address_s":"33 Commerce Valley Drive East Thornhill, ON L3T 7N6 Canada","_version_":1518135858631278592},{"id":"canon","compName_s":"Canon, Inc.","address_s":"One Canon Plaza Lake Success, NY 11042","_version_":1518135858640715776},{"id":"corsair","compName_s":"Corsair Microsystems","address_s":"46221 Landing Parkway Fremont, CA 94538","_version_":1518135858641764352},{"id":"dell","compName_s":"Dell, Inc.","address_s":"One Dell Way Round Rock, Texas 78682","_version_":1518135858641764353},{"id":"samsung","compName_s":"Samsung Electronics Co. Ltd.","address_s":"105 Challenger Rd. Ridgefield Park, NJ 07660-0511","_version_":1518135858642812928}]}}
{"responseHeader":{"status":0,"QTime":48,"params":{"q":"*:*","wt":"json"}},"response":{"numFound":11,"start":0,"maxScore":1.0,"docs":[{"id":"adata","compName_s":"A-Data Technology","address_s":"46221 Landing Parkway Fremont, CA 94538","_version_":1518135858246451200},{"id":"asus","compName_s":"ASUS Computer","address_s":"800 Corporate Way Fremont, CA 94539","_version_":1518135858502303744},{"id":"belkin","compName_s":"Belkin","address_s":"12045 E. Waterfront Drive Playa Vista, CA 90094","_version_":1518135858504400896},{"id":"maxtor","compName_s":"Maxtor Corporation","address_s":"920 Disc Drive Scotts Valley, CA 95066","_version_":1518135858511740928},{"id":"apple","compName_s":"Apple","address_s":"1 Infinite Way, Cupertino CA","_version_":1518135858281054208},{"id":"ati","compName_s":"ATI Technologies","address_s":"33 Commerce Valley Drive East Thornhill, ON L3T 7N6 Canada","_version_":1518135858631278592},{"id":"canon","compName_s":"Canon, Inc.","address_s":"One Canon Plaza Lake Success, NY 11042","_version_":1518135858640715776},{"id":"corsair","compName_s":"Corsair Microsystems","address_s":"46221 Landing Parkway Fremont, CA 94538","_version_":1518135858641764352},{"id":"dell","compName_s":"Dell, Inc.","address_s":"One Dell Way Round Rock, Texas 78682","_version_":1518135858641764353},{"id":"samsung","compName_s":"Samsung Electronics Co. Ltd.","address_s":"105 Challenger Rd. Ridgefield Park, NJ 07660-0511","_version_":1518135858642812928}]}}
{"responseHeader":{"status":0,"QTime":51,"params":{"q":"*:*","wt":"json"}},"response":{"numFound":11,"start":0,"maxScore":1.0,"docs":[{"id":"adata","compName_s":"A-Data Technology","address_s":"46221 Landing Parkway Fremont, CA 94538","_version_":1518135858246451200},{"id":"asus","compName_s":"ASUS Computer","address_s":"800 Corporate Way Fremont, CA 94539","_version_":1518135858502303744},{"id":"belkin","compName_s":"Belkin","address_s":"12045 E. Waterfront Drive Playa Vista, CA 90094","_version_":1518135858504400896},{"id":"maxtor","compName_s":"Maxtor Corporation","address_s":"920 Disc Drive Scotts Valley, CA 95066","_version_":1518135858511740928},{"id":"apple","compName_s":"Apple","address_s":"1 Infinite Way, Cupertino CA","_version_":1518135858281054208},{"id":"ati","compName_s":"ATI Technologies","address_s":"33 Commerce Valley Drive East Thornhill, ON L3T 7N6 Canada","_version_":1518135858631278592},{"id":"canon","compName_s":"Canon, Inc.","address_s":"One Canon Plaza Lake Success, NY 11042","_version_":1518135858640715776},{"id":"corsair","compName_s":"Corsair Microsystems","address_s":"46221 Landing Parkway Fremont, CA 94538","_version_":1518135858641764352},{"id":"dell","compName_s":"Dell, Inc.","address_s":"One Dell Way Round Rock, Texas 78682","_version_":1518135858641764353},{"id":"samsung","compName_s":"Samsung Electronics Co. Ltd.","address_s":"105 Challenger Rd. Ridgefield Park, NJ 07660-0511","_version_":1518135858642812928}]}}
{"responseHeader":{"status":0,"QTime":38,"params":{"q":"*:*","wt":"json"}},"response":{"numFound":11,"start":0,"maxScore":1.0,"docs":[{"id":"adata","compName_s":"A-Data Technology","address_s":"46221 Landing Parkway Fremont, CA 94538","_version_":1518135858246451200},{"id":"asus","compName_s":"ASUS Computer","address_s":"800 Corporate Way Fremont, CA 94539","_version_":1518135858502303744},{"id":"belkin","compName_s":"Belkin","address_s":"12045 E. Waterfront Drive Playa Vista, CA 90094","_version_":1518135858504400896},{"id":"maxtor","compName_s":"Maxtor Corporation","address_s":"920 Disc Drive Scotts Valley, CA 95066","_version_":1518135858511740928},{"id":"apple","compName_s":"Apple","address_s":"1 Infinite Way, Cupertino CA","_version_":1518135858281054208},{"id":"ati","compName_s":"ATI Technologies","address_s":"33 Commerce Valley Drive East Thornhill, ON L3T 7N6 Canada","_version_":1518135858631278592},{"id":"canon","compName_s":"Canon, Inc.","address_s":"One Canon Plaza Lake Success, NY 11042","_version_":1518135858640715776},{"id":"corsair","compName_s":"Corsair Microsystems","address_s":"46221 Landing Parkway Fremont, CA 94538","_version_":1518135858641764352},{"id":"dell","compName_s":"Dell, Inc.","address_s":"One Dell Way Round Rock, Texas 78682","_version_":1518135858641764353},{"id":"samsung","compName_s":"Samsung Electronics Co. Ltd.","address_s":"105 Challenger Rd. Ridgefield Park, NJ 07660-0511","_version_":1518135858642812928}]}}

Il a donc été possible de créer un cluster SolrCloud fonctionnel avec seulement 2 images Docker et 2 instructions bash.

Passage à Docker Compose

Deux commandes, c’est toujours trop, non ? Il est possible de décrire ce Solr distribué avec Docker Compose de la façon suivante :

zookeeper:
  image: jplock/zookeeper
  ports:
    - "2181:2181"
    - "2888:2888"
    - "3888:3888"

solr:
  image: solr
  links:
    - zookeeper
  command: /opt/solr/bin/solr start -f -z zookeeper

Avec ce fichier, SolrCloud se démarre avec une seule commande :

$ docker-compose up -d
Starting dockermyproject_zookeeper_1
Starting dockermyproject_solr_1

Il est toujours possible de scaler le cluster :

$ docker-compose scale solr=4
Creating and starting 2 ... done
Creating and starting 3 ...
Creating and starting 3 ... done

Creating and starting 4 ... done

La création de la collection se fait toujours de la même façon :

$ docker exec -it dockermyproject_solr_1 bin/solr create_collection -c myproject -shards 2 -replicationFactor 2

Connecting to ZooKeeper at zookeeper ...
Uploading /opt/solr/server/solr/configsets/data_driven_schema_configs/conf for config myproject to ZooKeeper at zookeeper

Creating new collection 'myproject' using command:
http://localhost:8983/solr/admin/collections?action=CREATE&name=myproject&numShards=2&replicationFactor=2&maxShardsPerNode=1&collection.configName=myproject

{
  "responseHeader":{
    "status":0,
    "QTime":18714},
  "success":{"":{
      "responseHeader":{
        "status":0,
        "QTime":17826},
      "core":"myproject_shard2_replica1"}}}

En revanche l’indexation de données pose un souci d’accès au cluster:

$ docker exec -it dockermyproject_solr_1 bin/post -c myproject example/exampledocs/manufacturers.xml
java -classpath /opt/solr/dist/solr-core-5.3.1.jar -Dauto=yes -Dc=myproject -Ddata=files org.apache.solr.util.SimplePostTool example/exampledocs/manufacturers.xml
SimplePostTool version 5.0.0
Posting files to [base] url http://localhost:8983/solr/myproject/update...
Entering auto mode. File endings considered are xml,json,csv,pdf,doc,docx,ppt,pptx,xls,xlsx,odt,odp,ods,ott,otp,ots,rtf,htm,html,txt,log
POSTing file manufacturers.xml (application/xml) to [base]
SimplePostTool: WARNING: Solr returned an error #500 ({msg=Error trying to proxy request for url: http://172.17.0.90:8983/solr/myproject/update,trace=org.apache.solr.common.SolrException: Error trying to proxy request for url: http://172.17.0.90:8983/solr/myproject/update? at org.apache.solr.servlet.HttpSolrCall.remoteQuery(HttpSolrCall.java:598)? at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:446)? at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:214)? at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)? at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)? at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)? at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)? at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)? at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)? at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextH) for url: http://localhost:8983/solr/myproject/update
SimplePostTool: WARNING: Response: <html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
<title>Error 500 {msg=Error trying to proxy request for url: http://172.17.0.90:8983/solr/myproject/update,trace=org.apache.solr.common.SolrException: Error trying to proxy request for url: http://172.17.0.90:8983/solr/myproject/update
        at org.apache.solr.servlet.HttpSolrCall.remoteQuery(HttpSolrCall.java:598)
        at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:446)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:214)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
        at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
        at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
        at org.eclipse.jetty.server.Server.handle(Server.java:499)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
        at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.NoRouteToHostException: No route to host
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:589)
        at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:117)
        at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
        at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
        at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
        at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
        at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
        at org.apache.solr.servlet.HttpSolrCall.remoteQuery(HttpSolrCall.java:565)
        ... 23 more
,code=500}</title>
</head>
<body><h2>HTTP ERROR 500</h2>
<p>Problem accessing /solr/myproject/update. Reason:
<pre>    {msg=Error trying to proxy request for url: http://172.17.0.90:8983/solr/myproject/update,trace=org.apache.solr.common.SolrException: Error trying to proxy request for url: http://172.17.0.90:8983/solr/myproject/update
        at org.apache.solr.servlet.HttpSolrCall.remoteQuery(HttpSolrCall.java:598)
        at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:446)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:214)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
        at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
        at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
        at org.eclipse.jetty.server.Server.handle(Server.java:499)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
        at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.NoRouteToHostException: No route to host
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:589)
        at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:117)
        at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
        at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
        at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
        at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
        at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
        at org.apache.solr.servlet.HttpSolrCall.remoteQuery(HttpSolrCall.java:565)
        ... 23 more
,code=500}</pre></p><hr><i><small>Powered by Jetty://</small></i><hr/>

</body>
</html>
SimplePostTool: WARNING: IOException while reading response: java.io.IOException: Server returned HTTP response code: 500 for URL: http://localhost:8983/solr/myproject/update
1 files indexed.
COMMITting Solr index changes to http://localhost:8983/solr/myproject/update...
SimplePostTool: WARNING: Solr returned an error #500 ({msg=Error trying to proxy request for url: http://172.17.0.91:8983/solr/myproject/update,trace=org.apache.solr.common.SolrException: Error trying to proxy request for url: http://172.17.0.91:8983/solr/myproject/update? at org.apache.solr.servlet.HttpSolrCall.remoteQuery(HttpSolrCall.java:598)? at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:446)? at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:214)? at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)? at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)? at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)? at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)? at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)? at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)? at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextH) for url: http://localhost:8983/solr/myproject/update?commit=true
SimplePostTool: WARNING: Response: <html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
<title>Error 500 {msg=Error trying to proxy request for url: http://172.17.0.91:8983/solr/myproject/update,trace=org.apache.solr.common.SolrException: Error trying to proxy request for url: http://172.17.0.91:8983/solr/myproject/update
        at org.apache.solr.servlet.HttpSolrCall.remoteQuery(HttpSolrCall.java:598)
        at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:446)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:214)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
        at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
        at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
        at org.eclipse.jetty.server.Server.handle(Server.java:499)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
        at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.NoRouteToHostException: No route to host
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:589)
        at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:117)
        at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
        at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
        at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
        at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
        at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
        at org.apache.solr.servlet.HttpSolrCall.remoteQuery(HttpSolrCall.java:565)
        ... 23 more
,code=500}</title>
</head>
<body><h2>HTTP ERROR 500</h2>
<p>Problem accessing /solr/myproject/update. Reason:
<pre>    {msg=Error trying to proxy request for url: http://172.17.0.91:8983/solr/myproject/update,trace=org.apache.solr.common.SolrException: Error trying to proxy request for url: http://172.17.0.91:8983/solr/myproject/update
        at org.apache.solr.servlet.HttpSolrCall.remoteQuery(HttpSolrCall.java:598)
        at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:446)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:214)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
        at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
        at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
        at org.eclipse.jetty.server.Server.handle(Server.java:499)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
        at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.NoRouteToHostException: No route to host
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:589)
        at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:117)
        at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
        at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
        at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
        at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
        at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
        at org.apache.solr.servlet.HttpSolrCall.remoteQuery(HttpSolrCall.java:565)
        ... 23 more
,code=500}</pre></p><hr><i><small>Powered by Jetty://</small></i><hr/>

</body>
</html>
Time spent: 0:00:24.074

Que ce passe-t-il ? Le port d’écoute de Solr n’a pas été rendu disponible depuis « l’extérieur ».

Une autre façon de décrire un SolrCloud distribué peut être envisagée :

zookeeper:
  image: jplock/zookeeper
  ports:
    - "2181:2181"
    - "2888:2888"
    - "3888:3888"

solr0:
  image: solr
  links:
    - zookeeper
  ports:
    - "8983:8983"
  command: /opt/solr/bin/solr start -f -z zookeeper

solrn:
  image: solr
  links:
    - zookeeper
  command: /opt/solr/bin/solr start -f -z zookeeper

Voici le nouvel essai de création et scalabilité du cluster :

$ docker-compose up -d
Starting dockermyproject_zookeeper_1
Starting dockermyproject_solr0_1
Creating dockermyproject_solrn_1

$ docker-compose scale solrn=3
Creating and starting 2 ... done
Creating and starting 3 ...
Creating and starting 3 ... done

La création de la collection est assez classique :

$ docker exec -it dockermyproject_solr0_1 bin/solr create_collection -c myproject -shards 2 -replicationFactor 2

Connecting to ZooKeeper at zookeeper ...
Uploading /opt/solr/server/solr/configsets/data_driven_schema_configs/conf for config myproject to ZooKeeper at zookeeper

Creating new collection 'myproject' using command:
http://localhost:8983/solr/admin/collections?action=CREATE&name=myproject&numShards=2&replicationFactor=2&maxShardsPerNode=1&collection.configName=myproject

{
  "responseHeader":{
    "status":0,
    "QTime":22830},
  "success":{"":{
      "responseHeader":{
        "status":0,
        "QTime":21188},
      "core":"myproject_shard2_replica2"}}}

Cette fois d’indexation des données fonctionne :

 $ docker exec -it dockermyproject_solr0_1 bin/post -c myproject example/exampledocs/manufacturers.xml
java -classpath /opt/solr/dist/solr-core-5.3.1.jar -Dauto=yes -Dc=myproject -Ddata=files org.apache.solr.util.SimplePostTool example/exampledocs/manufacturers.xml
SimplePostTool version 5.0.0
Posting files to [base] url http://localhost:8983/solr/myproject/update...
Entering auto mode. File endings considered are xml,json,csv,pdf,doc,docx,ppt,pptx,xls,xlsx,odt,odp,ods,ott,otp,ots,rtf,htm,html,txt,log
POSTing file manufacturers.xml (application/xml) to [base]
1 files indexed.
COMMITting Solr index changes to http://localhost:8983/solr/myproject/update...
Time spent: 0:00:01.236

Il est possible de vérifier la bonne réplication sur les autres noeuds du cluster :

for i in `seq 1 3`;
do
	docker exec -it dockermyproject_solrn_$i wget -q -O- 'http://localhost:8983/solr/myproject/select?q=*%3A*&wt=json'
done
{"responseHeader":{"status":0,"QTime":51,"params":{"q":"*:*","wt":"json"}},"response":{"numFound":11,"start":0,"maxScore":1.0,"docs":[{"id":"adata","compName_s":"A-Data Technology","address_s":"46221 Landing Parkway Fremont, CA 94538","_version_":1518137560790990848},{"id":"asus","compName_s":"ASUS Computer","address_s":"800 Corporate Way Fremont, CA 94539","_version_":1518137560893751296},{"id":"belkin","compName_s":"Belkin","address_s":"12045 E. Waterfront Drive Playa Vista, CA 90094","_version_":1518137560897945600},{"id":"maxtor","compName_s":"Maxtor Corporation","address_s":"920 Disc Drive Scotts Valley, CA 95066","_version_":1518137560918917120},{"id":"apple","compName_s":"Apple","address_s":"1 Infinite Way, Cupertino CA","_version_":1518137560958763008},{"id":"ati","compName_s":"ATI Technologies","address_s":"33 Commerce Valley Drive East Thornhill, ON L3T 7N6 Canada","_version_":1518137561087737856},{"id":"canon","compName_s":"Canon, Inc.","address_s":"One Canon Plaza Lake Success, NY 11042","_version_":1518137561100320768},{"id":"corsair","compName_s":"Corsair Microsystems","address_s":"46221 Landing Parkway Fremont, CA 94538","_version_":1518137561113952256},{"id":"dell","compName_s":"Dell, Inc.","address_s":"One Dell Way Round Rock, Texas 78682","_version_":1518137561132826624},{"id":"samsung","compName_s":"Samsung Electronics Co. Ltd.","address_s":"105 Challenger Rd. Ridgefield Park, NJ 07660-0511","_version_":1518137561141215232}]}}
{"responseHeader":{"status":0,"QTime":70,"params":{"q":"*:*","wt":"json"}},"response":{"numFound":11,"start":0,"maxScore":1.0,"docs":[{"id":"adata","compName_s":"A-Data Technology","address_s":"46221 Landing Parkway Fremont, CA 94538","_version_":1518137560790990848},{"id":"asus","compName_s":"ASUS Computer","address_s":"800 Corporate Way Fremont, CA 94539","_version_":1518137560893751296},{"id":"belkin","compName_s":"Belkin","address_s":"12045 E. Waterfront Drive Playa Vista, CA 90094","_version_":1518137560897945600},{"id":"maxtor","compName_s":"Maxtor Corporation","address_s":"920 Disc Drive Scotts Valley, CA 95066","_version_":1518137560918917120},{"id":"apple","compName_s":"Apple","address_s":"1 Infinite Way, Cupertino CA","_version_":1518137560958763008},{"id":"ati","compName_s":"ATI Technologies","address_s":"33 Commerce Valley Drive East Thornhill, ON L3T 7N6 Canada","_version_":1518137561087737856},{"id":"canon","compName_s":"Canon, Inc.","address_s":"One Canon Plaza Lake Success, NY 11042","_version_":1518137561100320768},{"id":"corsair","compName_s":"Corsair Microsystems","address_s":"46221 Landing Parkway Fremont, CA 94538","_version_":1518137561113952256},{"id":"dell","compName_s":"Dell, Inc.","address_s":"One Dell Way Round Rock, Texas 78682","_version_":1518137561132826624},{"id":"samsung","compName_s":"Samsung Electronics Co. Ltd.","address_s":"105 Challenger Rd. Ridgefield Park, NJ 07660-0511","_version_":1518137561141215232}]}}
{"responseHeader":{"status":0,"QTime":64,"params":{"q":"*:*","wt":"json"}},"response":{"numFound":11,"start":0,"maxScore":1.0,"docs":[{"id":"adata","compName_s":"A-Data Technology","address_s":"46221 Landing Parkway Fremont, CA 94538","_version_":1518137560790990848},{"id":"asus","compName_s":"ASUS Computer","address_s":"800 Corporate Way Fremont, CA 94539","_version_":1518137560893751296},{"id":"belkin","compName_s":"Belkin","address_s":"12045 E. Waterfront Drive Playa Vista, CA 90094","_version_":1518137560897945600},{"id":"maxtor","compName_s":"Maxtor Corporation","address_s":"920 Disc Drive Scotts Valley, CA 95066","_version_":1518137560918917120},{"id":"apple","compName_s":"Apple","address_s":"1 Infinite Way, Cupertino CA","_version_":1518137560958763008},{"id":"ati","compName_s":"ATI Technologies","address_s":"33 Commerce Valley Drive East Thornhill, ON L3T 7N6 Canada","_version_":1518137561087737856},{"id":"canon","compName_s":"Canon, Inc.","address_s":"One Canon Plaza Lake Success, NY 11042","_version_":1518137561100320768},{"id":"corsair","compName_s":"Corsair Microsystems","address_s":"46221 Landing Parkway Fremont, CA 94538","_version_":1518137561113952256},{"id":"dell","compName_s":"Dell, Inc.","address_s":"One Dell Way Round Rock, Texas 78682","_version_":1518137561132826624},{"id":"samsung","compName_s":"Samsung Electronics Co. Ltd.","address_s":"105 Challenger Rd. Ridgefield Park, NJ 07660-0511","_version_":1518137561141215232}]}}

Conclusion

Encore une fois, Docker Compose permet de monter rapidement et très simplement un environnement distribué. Il faut savoir que même si Zookeeper est une solution connue pour faire du Service Discovery dans un environnement Docker, ce n’est pas la solution la plus récommandée car la mise en place peut être complexe. Dans cet exemple, SolrCloud implémente lui-même l’intégration avec Zookeeper. Il n’y donc pas de sur-coût en terme d’implémentation.

Après avoir intégré un cluster de bases de données NoSQL et un moteur d’indexation distribué, quelle sera la prochaine étape ? 😛

https://blog.lecacheur.com/2015/11/18/solrcloud-avec-docker-compose/https://blog.lecacheur.com/wp-content/uploads/2015/11/Solr_Logo_on_white-1024x517.pnghttps://blog.lecacheur.com/wp-content/uploads/2015/11/Solr_Logo_on_white-150x150.png2016-12-12T09:19:04+01:00SeBDéveloppementLinuxcluster,docker,solrCet article sur la mise en place de SolrCloud fait écho à celui sur le cluster Cassandra avec Docker. Pour comprendre certains points abordés ci-dessous, il est nécessaire de connaitre la terminologie SolrCloud. Premiers essais avec Docker Comme précédemment, la première étape se résume à valider les images et la communication...SeBSébastien LECACHEUR23r0@laposte.netAdministratorLe weblogue de SeB

Camilo Herrera - septembre 18th, 2018 at 2:11 none Comment author #212069 on SolrCloud avec Docker Compose by Le weblogue de SeB

Thanks very good article, but I can not find the zkServer.sh, it is not in git 🙁

Répondre

Anonyme sur xmlstarlet – manipuler du XML en shell: “Merci bcq” Déc 25, 12:01

Selinge sur Simuler un fond vert pour une incrustation vidéo dans OBS: “OK c est bien” Déc 3, 11:29

Marieke sur Supprimer les accents UTF8 avec PHP: “Moi je cherche encore. Mon problème est de supprimer les émoticones ajoutées parfois par mes auteurs dans des recettes de…” Nov 14, 22:11

Alexandre TT sur Docker sous Windows (WSL 2) sans Docker Desktop: “Pour information, votre Windows 10 doit être en Build 1903 ou + en build 1809, les commandes wsl sont limitées…” Oct 20, 08:05

Premiers essais avec Docker

Passage à Docker Compose

Conclusion

1 Comment Already

Laisser un commentaire Annuler la réponse

SolrCloud avec Docker Compose

Premiers essais avec Docker

Passage à Docker Compose

Conclusion

Related Posts

Docker sous Windows (WSL 2) sans Docker Desktop

Revue de presse 2021 S44

Revue de presse 2021 S35

1 Comment Already

Laisser un commentaire Annuler la réponse