Open Source Projects

ceefour pushed to master at lumenitb/lumen

ceefour on GitHub - Thu, 01/29/2015 - 19:18
Jan 29, 2015 ceefour pushed to master at lumenitb/lumen
Categories: Open Source Projects

ceefour pushed to master at lumenitb/lumen-sdk

ceefour on GitHub - Thu, 01/29/2015 - 19:15
Jan 29, 2015 ceefour pushed to master at lumenitb/lumen-sdk
Categories: Open Source Projects

ceefour pushed to master at soluvas/soluvas-framework

ceefour on GitHub - Thu, 01/29/2015 - 14:19
Jan 29, 2015 ceefour pushed to master at soluvas/soluvas-framework
  • de4dad9 BC-2678 #resolve handle null. Sales Order List API Error
Categories: Open Source Projects

ceefour pushed to master at lumenitb/lumen

ceefour on GitHub - Thu, 01/29/2015 - 13:34
Jan 29, 2015 ceefour pushed to master at lumenitb/lumen
  • e720b48 Imports up to step 12. Fixed #11.
Categories: Open Source Projects

ceefour pushed to master at lumenitb/lumen

ceefour on GitHub - Thu, 01/29/2015 - 12:51
Jan 29, 2015 ceefour pushed to master at lumenitb/lumen
  • c8103ae Imports up to step 12. Fixed #11.
Categories: Open Source Projects

ceefour closed issue lumenitb/lumen#11

ceefour on GitHub - Thu, 01/29/2015 - 12:51
Jan 29, 2015 ceefour closed issue lumenitb/lumen#11 Import Yago2s database into Neo4j
Categories: Open Source Projects

ceefour pushed to master at lumenitb/FewDoubleCamera

ceefour on GitHub - Tue, 01/27/2015 - 14:30
Jan 27, 2015 ceefour pushed to master at lumenitb/FewDoubleCamera
Categories: Open Source Projects

ceefour pushed to master at lumenitb/lumen

ceefour on GitHub - Mon, 01/26/2015 - 16:07
Jan 26, 2015 ceefour pushed to master at lumenitb/lumen
  • bdce658 Import step 7 successful: 7. Import `yagoSimpleTypes.tsv`. 316 MiB, 5…
Categories: Open Source Projects

ceefour pushed to master at lumenitb/lumen

ceefour on GitHub - Mon, 01/26/2015 - 14:37
Jan 26, 2015 ceefour pushed to master at lumenitb/lumen
  • ab48384 Import step 6 successful. Refs #11.
Categories: Open Source Projects

ceefour commented on commit lumenitb/HumanDetection@1750fa0ac5

ceefour on GitHub - Mon, 01/26/2015 - 12:30
Jan 26, 2015 ceefour commented on commit lumenitb/HumanDetection@1750fa0ac5

masalah apa Pak @Marzuki001 ? kemungkinan besar setting proxy TortoiseGit sich.. yg harus dienable pas di ITB dan didisable pas di luar. (dia punya…

Categories: Open Source Projects

ceefour pushed to master at lumenitb/lumen

ceefour on GitHub - Mon, 01/26/2015 - 09:17
Jan 26, 2015 ceefour pushed to master at lumenitb/lumen
Categories: Open Source Projects

ceefour commented on commit lumenitb/HumanDetection@1750fa0ac5

ceefour on GitHub - Sun, 01/25/2015 - 23:17
Jan 25, 2015 ceefour commented on commit lumenitb/HumanDetection@1750fa0ac5

Very nice, @ahmadsyarif Makasih ya boss @vicknzr83 Pak @Marzuki001 trus ini diapain sekarang?

Categories: Open Source Projects

ceefour pushed to master at lumenitb/lumen

ceefour on GitHub - Sun, 01/25/2015 - 19:21
Jan 25, 2015 ceefour pushed to master at lumenitb/lumen
  • 6f15c1a Import successful up to step 4. Use tmpfs during import to get ~1000/…
Categories: Open Source Projects

ceefour pushed to master at lumenitb/lumen

ceefour on GitHub - Sun, 01/25/2015 - 17:55
Jan 25, 2015 ceefour pushed to master at lumenitb/lumen
  • 4812201 Import successful up to step 3. Use tmpfs during import to get ~1000/…
Categories: Open Source Projects

Answer by Hendy Irawan for create unique and merge slow

Hendy Irawan on Stack Overflow - Sun, 01/25/2015 - 13:41

Here are my experiences and tips:

  1. Use BatchInserter whenever possible. It requires the database to be offline and you structure code in restrictive way, but if you can do that you'll be rewarded.
  2. Put your database on RAM disk (tmpfs). This has about ~7.5x speed up on my system, from ~200 CREATE/s on HDD to ~1500 CREATE/s. For SSD speed up may be a bit less, but from my experience the HDD→SSD improvement is sadly not so significant for Neo4j. I'm in a tight spot because my DB is currently about 4 GiB and I only have 8 GB RAM, but even with potential virtual memory swapping tmpfs performance is much better than HDD+Linux buffers/cache.
  3. Limit Cypher query size, my sweet spot is 10-50 MERGE per Cypher query. Beyond 50 MERGEs, it gets slower. Around 500-1000 MERGEs in a Cypher query, Neo4j throws StackOverflowError.
  4. Use up to number_of_CPUs - 2 threads to run Neo4j transactions, leave 1 thread for main thread and another for Neo4j's own transaction writes. This results in ~95% CPU utilization across all 8 logical cores on my i7-3770K.
  5. Clear indexes before running, and create them after all operations have finished. When you need indexes, create your own in-memory (i.e. ConcurrentHashMap but ImmutableMap is better) but make sure you size your heap accordingly. Even if your heap goes into virtual memory, it's still faster than Neo4j's transactional flushes.
  6. If possible, create all indexed nodes beforehand. This means you don't have to care about uniqueness when you create relationships, and you can use MATCH instead of MERGE. For in-memory index, this means even if can't use ImmutableMap, you can use HashMap instead of ConcurrentHashMap even with multithreading.

Similar posts:

  1. Neo4j 2.0 Merge with unique constraints performance bug?
  2. How to improve performance for massive MERGE insert?
  3. Neo4jClient - Merge within ForEach with 1000 very slow (Unique Constraint)
Categories: Open Source Projects

ceefour pushed to master at lumenitb/lumen

ceefour on GitHub - Sun, 01/25/2015 - 10:54
Jan 25, 2015 ceefour pushed to master at lumenitb/lumen
  • 177b039 Use tmpfs during import to get ~1000/s. Refs #11
Categories: Open Source Projects

ceefour pushed to master at lumenitb/lumen

ceefour on GitHub - Sat, 01/24/2015 - 23:56
Jan 24, 2015 ceefour pushed to master at lumenitb/lumen
  • c64d2bc Import literals performance is "acceptable", starting from 2,282/s do…
Categories: Open Source Projects

ceefour opened issue neo4j/neo4j#3870

ceefour on GitHub - Sat, 01/24/2015 - 13:56
Jan 24, 2015 ceefour opened issue neo4j/neo4j#3870 Improve execute() Cypher error when node property in a param of CREATE is null
Categories: Open Source Projects

ceefour pushed to master at lumenitb/lumen

ceefour on GitHub - Sat, 01/24/2015 - 12:24
Jan 24, 2015 ceefour pushed to master at lumenitb/lumen
  • d200e56 Can import labels successfully for entire Yago (although some info ar…
Categories: Open Source Projects