SERP Hacker support forum - serposcope

Support forum for open-source softwares built by serphacker.com : serposcope the rank checker.

You are not logged in.

#1 2016-09-08 05:48:39

restre
Member
Registered: 2016-09-08
Posts: 7

Constant ERROR

Hi I am getting this constant error on the log and that specific keyword is not getting ranked. I added proxy's and captcha but error stil happening

What can I do to solve this?


[2016-09-08 00:41:26,436] [google-0] ERROR c.s.s.d.AbstractDB - SQL error
com.querydsl.core.QueryException: Caught JdbcSQLException for merge into "GOOGLE_RANK_BEST" ("GROUP_ID", "GOOGLE_TARGET_ID", "GOOGLE_SEARCH_ID", "RANK", "URL", "RUN_DAY")
values (?, ?, ?, ?, ?, ?)
    at com.querydsl.sql.DefaultSQLExceptionTranslator.translate(DefaultSQLExceptionTranslator.java:50) ~[serposcope.jar:na]
    at com.querydsl.sql.Configuration.translate(Configuration.java:459) ~[serposcope.jar:na]
    at com.querydsl.sql.dml.SQLMergeClause.executeNativeMerge(SQLMergeClause.java:511) ~[serposcope.jar:na]
    at com.querydsl.sql.dml.SQLMergeClause.execute(SQLMergeClause.java:298) ~[serposcope.jar:na]
    at com.serphacker.serposcope.db.google.GoogleRankDB.insertBest(GoogleRankDB.java:54) ~[serposcope.jar:na]
    at com.serphacker.serposcope.task.google.GoogleTask.insertSearchResult(GoogleTask.java:243) [serposcope.jar:na]
    at com.serphacker.serposcope.task.google.GoogleTask.onSearchDone(GoogleTask.java:193) [serposcope.jar:na]
    at com.serphacker.serposcope.task.google.GoogleTaskRunnable.run(GoogleTaskRunnable.java:115) [serposcope.jar:na]
    at java.lang.Thread.run(Unknown Source) [na:1.8.0_101]
Caused by: org.h2.jdbc.JdbcSQLException: Valor demasiado largo para la columna "URL VARCHAR(256)": "'https://www.davivienda.com/wps/portal/personas/davivienda/superior/filiales/!ut/p/b1/rZPJjptAFEW_xR-AKGZYYoZiNlAupo2FsRmMwdhgPH... (966)"
Value too long for column "URL VARCHAR(256)": "'https://www.davivienda.com/wps/portal/personas/davivienda/superior/filiales/!ut/p/b1/rZPJjptAFEW_xR-AKGZYYoZiNlAupo2FsRmMwdhgPH... (966)"; SQL statement:
merge into "GOOGLE_RANK_BEST" ("GROUP_ID", "GOOGLE_TARGET_ID", "GOOGLE_SEARCH_ID", "RANK", "URL", "RUN_DAY")
values (?, ?, ?, ?, ?, ?)  [22001-189]
    at org.h2.message.DbException.getJdbcSQLException(DbException.java:345) ~[serposcope.jar:na]
    at org.h2.message.DbException.get(DbException.java:179) ~[serposcope.jar:na]
    at org.h2.table.Column.validateConvertUpdateSequence(Column.java:329) ~[serposcope.jar:na]
    at org.h2.table.Table.validateConvertUpdateSequence(Table.java:737) ~[serposcope.jar:na]
    at org.h2.command.dml.Merge.merge(Merge.java:157) ~[serposcope.jar:na]
    at org.h2.command.dml.Merge.update(Merge.java:106) ~[serposcope.jar:na]
    at org.h2.command.CommandContainer.update(CommandContainer.java:78) ~[serposcope.jar:na]
    at org.h2.command.Command.executeUpdate(Command.java:253) ~[serposcope.jar:na]
    at org.h2.jdbc.JdbcPreparedStatement.executeUpdateInternal(JdbcPreparedStatement.java:157) ~[serposcope.jar:na]
    at org.h2.jdbc.JdbcPreparedStatement.executeUpdate(JdbcPreparedStatement.java:143) ~[serposcope.jar:na]
    at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeUpdate(ProxyPreparedStatement.java:61) ~[serposcope.jar:na]
    at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeUpdate(HikariProxyPreparedStatement.java) ~[serposcope.jar:na]
    at com.querydsl.sql.dml.SQLMergeClause.executeNativeMerge(SQLMergeClause.java:497) ~[serposcope.jar:na]
    ... 6 common frames omitted



Very sry, don't know how to put this in code...

Offline

#2 2016-09-08 06:30:14

Davide
Member
From: CH
Registered: 2016-03-24
Posts: 80
Website

Re: Constant ERROR

Im not shure but i think there is a bad config in your mysql database. some valuse cant be written because they are to long.


easy-develope.ch ¦ kontakt@easy-develope.ch

Offline

#3 2016-09-08 06:33:42

serphacker
Administrator
Registered: 2014-06-13
Posts: 411

Re: Constant ERROR

Thank you for the report,

No problem with your SQL database, it is a bug in serposcope the URL is too long and doesn't fit in the database. Will be fixed in next release.

Offline

#4 2016-09-08 06:36:09

Davide
Member
From: CH
Registered: 2016-03-24
Posts: 80
Website

Re: Constant ERROR

ok good to know.. one oftopic question. do you know when you release the next update with the bug fix of export range and other fixes?


easy-develope.ch ¦ kontakt@easy-develope.ch

Offline

#5 2016-09-08 06:37:34

serphacker
Administrator
Registered: 2014-06-13
Posts: 411

Re: Constant ERROR

I don't know for the release date, but it will be a small release with bugfix only, and yes it will include the export range fix.

Offline

#6 2016-09-08 06:45:20

restre
Member
Registered: 2016-09-08
Posts: 7

Re: Constant ERROR

Thanks guys!

Ok so what happens with the process when this error happens?

Offline

#7 2016-09-08 06:46:24

restre
Member
Registered: 2016-09-08
Posts: 7

Re: Constant ERROR

Oh and another thing I am getting this error with multithread.

[2016-09-08 01:39:51,872] [google-3] INFO  c.s.s.s.g.s.GoogleScraper - GOT status=[403] exception=[none]
[2016-09-08 01:39:51,872] [google-3] WARN  c.s.s.t.g.GoogleTaskRunnable - scrap failed for solicitud de compra de divisas because of ERROR_NETWORK
[2016-09-08 01:39:51,872] [google-3] WARN  c.s.s.t.g.GoogleTaskRunnable - no more proxy, stopping the thread
[2016-09-08 01:39:51,872] [google-3] INFO  c.s.s.t.g.GoogleTaskRunnable - google thread stopped

Why Is this?

Offline

#8 2016-09-08 06:47:28

serphacker
Administrator
Registered: 2014-06-13
Posts: 411

Re: Constant ERROR

The UrL is too long (966 characters) and is not recorded the database, the ranking is very likely to not be stored. I'll fix that.

Offline

#9 2016-09-08 06:48:40

serphacker
Administrator
Registered: 2014-06-13
Posts: 411

Re: Constant ERROR

restre wrote:

Oh and another thing I am getting this error with multithread.

[2016-09-08 01:39:51,872] [google-3] INFO  c.s.s.s.g.s.GoogleScraper - GOT status=[403] exception=[none]
[2016-09-08 01:39:51,872] [google-3] WARN  c.s.s.t.g.GoogleTaskRunnable - scrap failed for solicitud de compra de divisas because of ERROR_NETWORK
[2016-09-08 01:39:51,872] [google-3] WARN  c.s.s.t.g.GoogleTaskRunnable - no more proxy, stopping the thread
[2016-09-08 01:39:51,872] [google-3] INFO  c.s.s.t.g.GoogleTaskRunnable - google thread stopped

Why Is this?

Your proxy is blacklisted (403 status) or has an issue. Try to use more proxies or reduce/disable multithreading and increase pause time.

Offline

#10 2016-09-08 06:58:57

restre
Member
Registered: 2016-09-08
Posts: 7

Re: Constant ERROR

Yup that solved it! You tha man!!!

Offline

#11 2016-09-08 08:36:23

restre
Member
Registered: 2016-09-08
Posts: 7

Re: Constant ERROR

HI! One more question...

I have anti-captcha configured and 5 working proxys..

What would be the best configuration for #pages, results per page, pause times, threads and fetch? To have an ideal working configuration?

Thx again!

Offline

#12 2016-09-08 08:50:07

serphacker
Administrator
Registered: 2014-06-13
Posts: 411

Re: Constant ERROR

the default configuration is the best. 1 page of 100 result, 1 thread every 2 proxy (so if you have 5 proxys, I would setup 2 or 3 threads), regarding the pause time, greater is better it really depends how fast you want the run to be done, but if it is faster, you increase captcha and rate limit risk.

Offline

#13 2016-09-08 13:17:03

restre
Member
Registered: 2016-09-08
Posts: 7

Re: Constant ERROR

Hi! In an exported CSV, what do the "NULL" results mean??

Offline

#14 2016-09-08 13:42:32

Davide
Member
From: CH
Registered: 2016-03-24
Posts: 80
Website

Re: Constant ERROR

restre wrote:

Hi! In an exported CSV, what do the "NULL" results mean??

it means there was no ranking detected or there was an error by crawl this keyword at the specific day.

Last edited by Davide (2016-09-08 14:03:23)


easy-develope.ch ¦ kontakt@easy-develope.ch

Offline

#15 2016-09-08 13:58:47

restre
Member
Registered: 2016-09-08
Posts: 7

Re: Constant ERROR

Thanks!!!

Offline

Board footer