Migration to New Board Software

[dstar]

Why instead? The project can be simply double licensed (as many other projects are), eg CC-nc/GPL (or OSL).

Because I'm not as familiar with how double licensing works and so don't think about it?

Actually, that would make sense.

Shalon Wood

[Shadowhawk]

Because I'm not as familiar with how double licensing works and so don't think about it?

The only problem with double licensing I can think of off-hand is contributions under only one of the licenses, which can lead to fork, or fork-like state.

[Spec8472]

Let me be the first to suggest you learn Perl -- not neccessarily for _this_ project, but just in general. My last two jobs were gained on the strength of my perl ability, and my current job (which is 100% perl coding) pays...well, better than I've ever dreamed of making. It's a damned good skill to have.

I'd like to get a Comp. Sci degree too, but that's just not going to happen for a few years down the road.  I understand that perl is a powerful language in its own right - it just doesn't fit in with my current plans or needs.
The time I'd spend learning perl, I'd rather be spending learning how to make better use my camera.

I've got a fairly stable job at the moment (as far as jobs for a 'Blue Chip' company go), which involves developing database-oriented .NET applications, amongst an array of other things.

Could I convince you to use the GPL instead? Not because I'm a GPL bigot, but because a company that's going to use this for commercial purposes is likely to put more effort into making it do what they need, and a non-commercial license would stop them from using it.

Not at the present.
If someone wants to use it commercially, they're welcome to contact me to negotiate terms of use.

If they're doing it as a 'free service' for someone (or their own internal use) I'd be more than happy to give them a licence to use it.

If they're going to be charging customers to convert forums, then (depending upon circumstances) they'd need to provide an investment into the project - either monitary (which I'd put towards further development, hosting, or similar), or development of the project itself (extensions, more flat-text input formats, whatever).  

The GPL (and other similar licences), as far as I can see, doesn't stop someone from using the software/source code making a hojillion dollars out of using it (not that I think this is seriously going to make anyone any sort millionaire), without providing any sort of input into the project.

I don't want to sound like I'm being greedy, it's all about give and take -- I provide this webspace, because I enjoy reading Fel's works and reading/participating in the various threads on the board.  
I believe that if a company wants to take the conversion software I wrote, they have to give something back if they want to make money out of it.

[Shadowhawk]

The GPL (and other similar licences), as far as I can see, doesn't stop someone from using the software/source code making a hojillion dollars out of using it (not that I think this is seriously going to make anyone any sort millionaire), without providing any sort of input into the project.

The GPL is about two things:

• derivative code has to be covered under the same license (that e.g. means if they make your converter part of their forum, the forum has to be under GPL as well)
• If they publish the GPL-ed program (and they are not developing it only not for internal use) , they have to make source code available for free as well (to those who get the program itself)

This should mean that all the improvements they put into the code are available for all. I.e. the return is in the development of the program (unless they don't publish their version at all). It is not a BSD-type license... - licenses

[Spec8472]

OK, Update:
I've spent most of today on the conversion tool.

A few problems remain however, but at the very least - all of the content is there and converts across quite neatly.

There is a semi-functional version of the new forum here: http://home.plebian.net/forum-phpbb/

NOTE: Whilst you can login (your password is the same), you may be redirected to http://www.arweneventstar.com/ or recieved a "No such server" error if you do so. Navigate back to the forum and hit refresh to see your login.

This issue is Only for the testing platform.

NOTE: Don't change any account settings on the test forum, as you WILL lose any changes.


Known Issues:
* Converted Threads (Topics) are listed in reverse order (By TopicID, instead of Last Post Date)

* "Moved" threads arn't converted correctly yet (requires some more processing to find the 'destination' thread)

* [s]BBCode doesn't convert properly[/s] (Updated: 17/01 - Now fixed), [s]and HTML is displayed instead of being interpreted.[/s] (HTML Issue Fixed - PHPbb Config issue)

* Links referring to the old forum don't convert (Updated: 18/01 - Partly Completed, untested. The converter now generates a new YaBB.pl file which does a HTTP (server-side) redirection to the correct new URL, and if available: board and/or thread) -- Perl Geeks see below post.

* [s]Signature "Carriage Returns" show up as "&&" [/s] (Updated 17/01: Fixed)

* Avatars don't carry across (neither 'internal' nor 'external')

* 'Sticky' threads arn't 'sticky' after conversion. (I didn't even think of importing those settings, oops)

* [s]Ranks (different titles based on position/# posts) don't get applied to people properly[/s] (Fixed, 17/01)

* Moderators and Permissions don't get converted

* [s]Post Counts for users don't get converted[/s]. (Fixed, 17/01)

* [s]Search, as yet, isn't Working - this is because of another issue (inside the app itself).[/s] (Updated 17/01 - Now fixed, initial testing shows it takes a darn long time to build the word-index)

* Private Messages may or may not convert correctly (Untested)

...and probably some more bugs too...

I want to fix this up first before doing the real conversion - so it'll be another few days before it all goes ahead.

[s]It's also far faster at conversion than the provided (really broken) conversion script on the PHP Site - it takes only about 20-30 seconds to convert around 3,500 posts, on just under 300 topics, for 3 boards, plus all users (and a good chunk of the search indexing too). That's around 10x faster than the PHP Script. The tradeoff is that it uses a chunky amount of memory - for the above conversion, about 48MB. [/s]

(Updated 17/01) The current conversion time is around 8 minutes for the current forum (about 7 minutes of that time is spent doing the full-text indexing), and using ~80MB of memory.

[Shadowhawk]

Spec8472: take your time with writing conversion software, converting itself... and maybe adding features known from YaBB to PHPbb, or modifying/configuring PHPbb to match old forum more closely (ranks names, ranks positions (i.e. numbers of posts),...). Or using already written PHPbb mods (you can find plenty of them at sourceforge.net and in phpBB site itself; Fully Modded phpBB is version of phpBB forum with many mods added... but you probably would want to chose the modules).

BTW. are you planning to use PHPbb 1.x or PHPbb 2, Spec8472?

[Lochar]

Spec8472: take your time with writing conversion software, converting itself... and maybe adding features known from YaBB to PHPbb, or modifying/configuring PHPbb to match old forum more closely (ranks names, ranks positions (i.e. numbers of posts),...).

Yeah, I don't want to have to post 500+ times again just to have a fun title.  *grin*

[Spec8472]

Spec8472: take your time with writing conversion software, converting itself...

Thanks.

and maybe adding features known from YaBB to PHPbb, or modifying/configuring PHPbb to match old forum more closely (ranks names, ranks positions (i.e. numbers of posts),...). Or using already written PHPbb mods (you can find plenty of them at sourceforge.net and in phpBB site itself; Fully Modded phpBB is version of phpBB forum with many mods added... but you probably would want to chose the modules).

I'm converting to stock-standard PHPbb 2 at the moment. I'll look into supporting additional mods.

BTW. are you planning to use PHPbb 1.x or PHPbb 2, Spec8472?

PHPbb 2. This is the latest version available on the PHPbb site, and so far as I can tell -- will recieve security updates and such.

[Spec8472]

Hey, I've updated the 'progress' post just up there ^^ (points up a few posts).

If you're a perl geek, tell me if there's a better way of doing the below script.

I can write the 'board' and 'forum_id', etc to arrays/whatever... I just don't have a clue how to implement a lookup like that in perl.  (Plus, I can't find a decent perl reference).

Code: Select all

#!/usr/local/bin/perl 
// YaBB Redirection Script. 
// Redirects the user to the new converted PHPBB Forum, and if possible - the correct thread/forum requested. 



// Dissection of a typical direct link to a thread
// YaBB.pl?action=display;board=news;num=1104326575;start=15#17
//
// Parameters: 
// action: what action to take -- for links this is always display
// board:  what board this thread belongs to (this is a hint for YaBB - not necessary if we know the new thread ID) 
// num: the YaBB thread 'number' - this will be completely different to the PHPBB Thread ID
//  start: the thread's post number to display
// 
// if num is present, and we know what it is then redirect to the new phpbb host, with the new Thread ID. 


// Dissection of a typical direct link to a board
// YaBB.pl?board=news
// 
// Parameters: 
// board: What board to show -- this will be completely different to the PHPBB Forum ID. 
// 
// if board is present, and we know what the new Board ID is, then redirect to the new phpbb host, with the new Forum ID. 

// NOTE: 
// The Conversion Software will write out each of the below sections - one copy for each thread and forum converted. The below variables will be replaced before writing. 
// THREAD.PHPBB_ID = The PHPBB Row Identifier for a given thread
// THREAD.YABB_ID = The YaBB Textfile Identifier. 
// FORUM.YABB_ID = The YaBB board identifier. 
// BOARD.PHPBB_DOMAIN = The PHPBB Domain as configured. 
// BOARD.PHPBB_PATH =  The PHPBB Path as configured. 


$done = 0; 

my %replacement_Threads = ("THREAD1.YABB_ID" => "THREAD1.PHPBB_ID", "THREAD2.YABB_ID" => "THREAD2.PHPBB_ID"); 
my %replacement_Boards = ("BOARD1.YABB_ID" => "BOARD1.PHPBB_ID", "BOARD2.YABB_ID" => "BOARD2.PHPBB_ID"); 


// If a thread has been specified, try redirecting to it.  
if (param("num")) { 
      $num = param("num"); 
      print "http://BOARD.PHPBB_DOMAIN/BOARD.PHPBB_PATH/view.php?t=$replacement_Threads{$num}

"; 
      $done = 1;  }; 

// if a thread has not been specified, but a board has - try redirecting to that board. 
if (param("board") && $done = 0) { 
      $board = param("board");
      print "Location: http://BOARD.PHPBB_DOMAIN/BOARD.PHPBB_PATH/view.php?f=$replacement_Boards{$board}

";   
      $done = 1;  }; 

if ($done = 0) { 
      print "Location: http://BOARD.PHPBB_DOMAIN/BOARD.PHPBB_PATH/

";   
      $done = 1;  }; 

[dstar]

Hey, I've updated the 'progress' post just up there ^^ (points up a few posts).

If you're a perl geek, tell me if there's a better way of doing the below script.

I can write the 'board' and 'forum_id', etc to arrays/whatever... I just don't have a clue how to implement a lookup like that in perl.  (Plus, I can't find a decent perl reference).

What is it you want to change?

Shalon Wood
(pre-caffiene....)

[Shadowhawk]

Hey, I've updated the 'progress' post just up there ^^ (points up a few posts).

If you're a Perl geek, tell me if there's a better way of doing the below script.

I can write the 'board' and 'forum_id', etc to arrays/whatever... I just don't have a clue how to implement a lookup like that in Perl.  (Plus, I can't find a decent Perl reference).

Thanks for the update on the progress of board migration from YaBB (Perl+flat files) to phpBB (PHP+database).

I'm not a Perl geek, although I wrote some simple Perl scripts. Unfortunately, none for the web, and none using databases. Nevertheless your Perl code looks like stub (i.e. comments describing what you want to have done), not the code itself. If nobody else with greater Perl experience from forum readers (or IRC lurkers) will come forward, I'd try to allocate some time for that this weekend.

By the way, could you mail me (e-mail in profile), or PM me on IRC the password for new forum version? I'd like to check the differences (and try to find modules to eliminate them). Thanks in advance.

<hr>
Finally, about Perl references:

Unfortunately, I seem to lost the bookmars with Perl references I used to learn it myself. I'd recommed paper books (or their electronic versions): "Learning Perl", "Programming Perl" (the Perl encyclopedia) and "Perl Cookbook" (with lots of example code). I learned Perl mainly from documentation included with Perl distribution, namely "Perl Lessons" (link to online version) and perl manuals, beginning with perlintro(1). And of course take a look at http://www.perldoc.com.

Lately there was Perl course in Linux Gazette: part 1, part 2, part 3, part 4, part 5. And I found "Perl 5 by Example" by David Medinets.

Like usual, Google is your friend.

<hr>
By the way, why in phpBB we can't have exactly the same numbers/IDs of articles as in current YaBB version?

[Spec8472]

What is it you want to change?

I was thinking that writing out a complete copy of the redirection code for each thread is (probably) going to be slow to interpret for several hundred threads.

Larger converted boards may have far more threads.

Nevertheless your Perl code looks like stub (i.e. comments describing what you want to have done), not the code itself.

The last 10 or so lines are the code that (I think) should work at doing the redirection.

If nobody else with greater Perl experience from forum readers (or IRC lurkers) will come forward, I'd try to allocate some time for that this weekend.

Thanks, I'll try and be online as much as possible then.

By the way, could you mail me (e-mail in profile), or PM me on IRC the password for new forum version? I'd like to check the differences (and try to find modules to eliminate them). Thanks in advance.

What password are you referring to? Your account on the forum itself has the same password as your current password here.

Finally, about Perl references:

Ah, thanks for those.

Like usual, Google is your friend.

Tried that, didn't get far :/
Maybe I'm not using the right keywords (fairly likely), but I'm used to getting decent results for my queries.

It took a while to get a result with perl and google.

By the way, why in phpBB we can't have exactly the same numbers/IDs of articles as in current YaBB version?

PHPbb uses the selected databases auto-incrementing integer datatype as its row key for boards, threads, posts, etc. I'd rather not try and modify these IDs, for fear of inducing conflicts and such.

[dstar]

I was thinking that writing out a complete copy of the redirection code for each thread is (probably) going to be slow to interpret for several hundred threads.

Larger converted boards may have far more threads.

Ah! I think I understand (my sinuses are _killing_ me today, so let me know if I'm wrong).

If I understand correctly, what you want is a hash:

my %redirect_map = ( "url_for_old_location_thread1" => "url_for_new_location_thread1", "url_for_old_location_thread2" => "url_for_new_location_thread2",);

Then you'd do

print "Location: $redirect_map{$url}
";

assuming that $url contained the requested url.

Shalon Wood

[Spec8472]

print "Location: $redirect_map{$url}
";

assuming that $url contained the requested url.

Neat! OK, modified the above code.


Edit:
More progress made, focusing on Avatar conversion at the moment.

[Shadowhawk]

I have browsed through modules/hacks database for phpBB, which is to be new forum engine, at http://www.phpbb.com and http://www.phpbbhacks.com, searching for modules which will make available the features from current YaBB engine.

First of all, try EasyMOD for installing phpBB modules, available at Area51. (Why the heck they require to be registered on this forum to download EasyMOD, is beyond me).

On http://www.phpbb.com  I have found:

• In Add-On category: "Top 'X' Posters" (like now in Members page in YaBB), "Custom Title MOD" (for MommyDoom, Blyker and JMan-5 - it is called  Position in YaBB Member List).
• In Admin Tools category "Rank Images Display Multi. Times", which I think would give what we have now: number of stars depending on rank. Maybe, I haven't tried it.
• In Cosmetic/Styles category: "Rank in memberlist" (I think it is what is called Position in YaBB Member List), maybe "Pagination MOD" for easier navigation, "Post icons" (like Message icon in YaBB - maybe totally unnecessary), "Today At/Yesterday At", "Last Topic MOD" (Last Updated Topics in the YaBB, index page), "Auto Subject On Reply" to be more close to what YaBB does (defaults to "Re: <topic title>" like in YaBB, not empty title like in phpBB), "Link Poster's Name To Profile", "Next/Previous topic link below post", "Register from Login Prompt" (maybe it isn't present in YaBB, but it is certainly something worth having).

On http://www.phpbbhacks.com I have found (there is search hacks field on this site, so finding mods by name should be easy) "Memberlist Sortable by Letter" (like View by Letter in Member List in YaBB) and maybe "Memberlist Find User" for when the forum will grow up in users, "Gender Hacks" (although most users don't provide it), maybe "Full Pagination" (for undivided pagination: either that or mentioned above "Pagination Mod") and "Keep Unread Flags". I have not mentioned hacks for implementiong for example YaBB [s]strikethru[/s] code. And I have propably ommited some...

And of course there are phpBB solutions with some/selected modules installed, like Fully Modded phpBB, IntegraMOD or EclipseBB (beta version)...

Because for now test page for new forum engine, http://home.plebian.net/forum-phpbb/, is "broken" (i.e. it does not contain posts/threads from here, and users from here), I cannot test what additional hacks/mod would be useful to make all the functionalities/features from here present in new, phpBB based, Sennadar Forum. Keep in mind that we will have many of the features absent here, and in addition I hope for better scalability and faster service.