Tobias Schlitt - a passion for php

A little bit belated, but as promised during the session, I just uploaded my slides from the talk 6 essential PHP development tools in 60 minutes. Since there is so much to say I actually managed to get 4,5 tools through and for all of these it felt to be too few time. I think I should provide a workshop for the next conference

Hope to see you there! 'till next time.

A week ago Sebastian pointed out an article on LinuxJournal, which talks about documentation coverage. By the question "Isn't that exactly what tobyS' tool does?" I felt remembered, that I wanted to blog the little tool I wrote for eZ Components a while ago. Since this blurb was lurking in my blog for another week, you get my writings a little more belated.

The actual idea was inspired by a blog post by Lukas Smith which threw the term "documentation coverage" into my mind. We (as in "the eZ Components team") are very keen on documentation, which is reflected in extensive API docs, additional tutorials and lots of example code. While the latter 2 are still conveniently checkable manually, API docs are not that easy to validate, resulting from the huge number of classes and class members in eZ Components. Typos, missing doc-tags and violations of our documentation standards are not easily detectable and can occur easily during development. Checking every doc block by hand is a live time work and even if you try to do so, you will miss many small issues.

Therefore I wrote a little tool to assist us with checking the consistency of API documentation. The tool uses PHPs reflection API to retrieve the OO elements of a component and uses a simple (regex based) parser to extract the doc block elements assigned to them. The pure parsing already gives a hint on broken documentation tags, as far as this is possible using a regex based parser. A simple visitor interface can be used to perform checks on the tree of API-elements and their documentation.

While (almost) 100% of eZ Components API elements are documented using a phpDocumentor syntax. So our major concern is not the doc coverage itself, but syntactical correctness of the phpDocumentor annotations and (as far as possible) the semantical correctness. While the first subject can be checked quite easily using a "real" parser (and even with the current one), the latter one is quite tricky, as very semantical check is.

The current implementation checks especially for the availability of certain tags for certain elements (like a @package for each class) and correct values for annotations like @copyright. Beside that it checks if all parameters of a method are documented, if the documented types match eventually available type hints and if the order is correct. Although this sounds not so much work to be performed, we were amazed how many small and bigger issues with the documentation were already found this way.

Since this proof-of-concept implementation works quite good, I started implementing a real parser for the docs, to get a better tree-structure and perform more valuable checks. But this is at a very early stage and not publically available, yet. Anyway, although the current main-script is very eZ Components specific, the whole thing might by valuable to others, too, which is the main reason for this blog post.

You can check the script out of our SVN, where it is called docanalysis.php. Adjusting this stuff to your own project should be easy. Hope this is valuable for someone.

Are you using my phpDocumentor for VIM plugin? Go ahead and stack it! :)

And don't forget to stack eZ Components, kPortal, PHPUnit, Xdebug and all the other PHP projects, available on Ohloh!

I recently added 2 new functions to my PHP FTPlugin for VIM, which I wanted to implement for a longer time and recently needed quite often:

PhpAlign()

Often you have written down an array declaration or a set of variable
assignements. Usually things look somewhat ugly the, like

Aligning this definition properly is an ugly, boring work. The PhpAlign()
function takes it from you and aligns the array declaration properly:

This also works with usual variable assignements: ::

becomes

PhpUnComment()

Often you want to comment or un-comment a couple of lines, because you
currently change those and want to make a backup or simply want to bring
alternative code in place. For multiple reasons you may not want to use
multi-line commens for this (e.g. because you the closing sequence inside the
code or because they simply look ugly. PhpUnComment() simply comments a line
which is not commented and un-comments a line that is commented.

Selecting these lines (all of them) and running PhpUnComment() results in:

Both function are mapped to shortcuts in visual mode (<Ctrl>-a for PhpAlign() and <Ctrl>-c for PhpUnComment()), if you are using my FTPlugin.

The changes are already included in my SVN and documented there, too. Do get a recent checkout do:

$ svn co svn://svn.toby.phpugdo.de/PDV

If you kept an eye on Planet-PHP during the past days, you will have seen a discussion emerging about the sense of doc blocks and documentation in general. Travis Swicegood states a quite interessting point of view. His main point is, that program languages themselves are declarative and that it is somewhat stupid to describe the purpose of a method in plain english, if it has been already described by the programming language itself. Although I never thought in this direction so far, I can see a valid point here. Partly.

I basically agree with Travis, that programming languages are declarative and (for the usual case) you shouldn't need much documentation for a class, if you name your methods and attributes properly. At least for an experienced programmer, the names should be obvious in most cases. But at least in PHP we have some major issues, which are modelled in code and are not obvious from the outside:

The most obvious thing are exceptions. Unlike Java, PHP does not require that an exception is either caught or the method is declared to throw it again. If you are going to use a programming libraray, you therefore have no possibility to see, which method throws which exceptions, without proper documentation. Another point are read/write only properties. Since PHP does not have a modifier for this so far, you need to go the way of interceptors. Properties which are "declared" using overloading are not as obvious as real attributes are, because they simply are not a declarative construct of the language itself. You therefore need documentation here to mark those up for the user.

But even if you neither use exceptions, nor interceptors to define properties, you still have a lot of cases, where pure naming is not sufficient to make a foreign user see in 1 glance, what happens. Simple things like the classes that are used in realtion with another class are often not obvious. Just think in matters of the factory pattern, dependency injection and similar. Additionally, if you only have the code itself, navigation is quite hard, if you have a large code base. Browsing directory structures to look for what you are searching is not fun, especially, if concepts are used, that you are not familiar with, yet.

Another (and propably the most important) point is simply the difference between people. I'm doing PHP for over 6 years now, I've dealt with a lot of libraries in PHP and have developed many applications. I'm heavily affected by what I've done and I've been inspired by a lot of people and experiences. But you are different! Are you sure, that my thoughts are as obvious to you as they are to me? No, you never can asure that. And that's a major point, why I will have to explain my thoughts to you in a more complex syntax than PHP to make you understand the overall concept behind my doings. This is quite natural and will not change any time soon, because even if programming languages are declarative, they only hava a very limited syntax. Surely you can describe all that in a programming language, too (you do! what else is your complete code base than your concepts and thoughts written down?). But to understand it all, you would need to read the complete code and not only the prototypes.

I think I made my point quite clear so far. But Travis still has a valid point. Inline API docs are not, what he is looking for in the matters of "documentation". I agree with him. No, I'm not contradictory, but in the matter of my last paragraph, people are simply different. While I consider inline API documentation an important part of any code, because it simply gives me an overall reference of what is available, I don't look at in the first glance.

While thinking about Travis matters, I analyzed my own way of working with documentation. When I start with something new (a new library for example), the first thing I want to know is "How does this work in general?". I'm sure a lot of developers have the same feeling and don't want to dig into API docs for hours and hours, before they have the big picture and understand the overall concept.

So, what is basically needed to get started is a tutorial. I consider tutorials the most important part of any documentation. A simple step by step introduction to a practical usage example gives you more than 1000 lines of inline docs when getting started with something new, because you gather the main points you are interessted in with very few learning effort:

• What is this?
• What can I do with it?
• What is the overall concept behind it?
• How am I expected to use it?

These 4 questions reflect the main information I typically want to have when digging into something new and I actually cannot imagine anything that can help me better than a tutorial here.

Ok, after reading the tutorial for a specific component, what comes next? The next step is to actually face the problem I want to solve using the library. Yes, that is basically what I want to do, solving a problem. I usually start copying some code from the tutorial, which I consider valid for the solution of my problem. Now I have to adjust that piece of code to suite my needs and make it do, what I actually want it to do. At this point API documentation enters the game. What I need is a reference of a specific class or method, to see with 1 glimpse, what I can do to adjust the behaviour. The most important questions now are:

• What are the options of an object and how do they affect the objects behaviour?
• What are the parameters of a method and how do they affect the behaviour of the method?
• What are related classes and methods, which could propably solve my problem easier or in a better way?
• Which side effects will I have to expect, if I use this class / method?

While the first 2 points should mostly be covered by the code itself (you know, programming languages are declarative), the latter 2 are mostly not. They show another 2 important points of API documentation: The relation between classes/methods and some portions of the background information, that is usually hidden somewhat deep inside the code.

So far so good, I managed to realize what I wanted to achieve. If I reached this point with reasonable time effort, the documentation of the project seems not that bad, does it? What comes next? I propably will not use the same classes I just used very soon again. But the time will come, when I come back for one of 2 reasons:

• The code I've writen needs maintainance
• I need to solve a similar problem and want to use the library again

What is different now in contrast to the first time I needed the documentation? I already have the big picture of the component in mind. I also have some working code at hands, which was written by myself and should therefore be understandable for me (inline docs help! ;). The tutorial might be helpfull now, to recall some facts. But more important are the API docs now, again.

So, why am I telling you all this? Basically to show you my view on the documentation issue. I agree with the fact, that API docs are not the solution to all of your problems. A well written tutorial helps much more in the first steps, than any API docs can do. And I think that is basically the same point Travis mentions, when he says "Give me a unit test any day over a three paragraph docblock". Exploring new terrain by example is much more effective than digging into pages of API description. But that does not mean, API docs are useless. They are very important for the further steps and should solve the issue of the limited declarativity a programming language has, due to its limited syntax.

Finally I'd like to refer to the documentation of eZ Components. I think we manage very good to provide all of the named. For each component we have different documentation forms online. A tutorial for the main functionality of a component exists (for example the ConsoleTools tutorial). Beside that, we have additional code examples shipped with each components source, for people that prefer reading PHP over English. ;) The complete source of eZ Components is inline documented and the documentation is rendered online (e.g. ConsoleTools). The API docs are enhanced by a lot of usefull features, like a special markup for the most important classes and some example code in the class docs of. Also quite nice in my eyes is the linking between tutorials and API docs, which helps you to get detailed information about a specific class or method right from the tutorial with 1 click. Finally, especially important for Travis ;), eZ Components are fully unit tested.

Is there anything more you would expect? Tell me, I'm currious! :)

More than a half year after my last "Comfortable PHP editing with VIM" post, I take up this series again, although I decided to let it die in January. Sadly I did not find any time by now, to extend PDV (the PHP Documentor for VIM plugin) further than it's version 1.0.1, which is available through vim.org and on my SVN. Anyway, I collected a lot of (at least for me) helpful VIM tips, which I'd like to share here:

Next occurrence of a word

If you hit * in command mode and your cursor resides on a word, you are taken to the next occurrence of the word. This is quite nice, if you like to know, where a function is called again.

Find matching brace

VIM 7.0 luckily highlights matching parenthesis, if your cursor resides on a brace, but sometimes you need to quickly jump to that matching brace. You can achieve this by hitting the % sign in command mode.

Repeat the last change

It often occurs, that you need to perform 1 change several times, but not often enough to write a short script or to address the changes with a complex regex. In those cases you can perform the change once, move the cursor to the next place and hit the . (dot) char, in command mode.

Enhanced functionality with external programs

This tip is quite commonly known, but I repeat it here, because it's really helpful: Although VIM has a huge set of useful functions, it does not have everything. If you don't find a utility in VIM, you can simply use extern shell tools to perform certain tasks, by using :!<command> in command or visual mode. For example sorting a range of lines can easily be achieved with the following 2 steps: 1. Select the lines to sort in visual mode. 2. Type :!sort and hit return.

Indenting and unindeting

Especially when editing code, it happens often, that you have to indent/unindent a code block. If you use VIM's auto-indenting features (like used by my .vimrc), VIM can do that job for you, too. Simply select the lines to (un-)indent in visual mode and hit <} to unindent 1 level or > to indent 1 more level. This also works (as usual in VIM) with quantifiers like 4> to indent 4 more levels.

Remote file editing

Often you have to quickly edit a file on a remote server. Usually you SSH to that server or mount a remote file system locally, edit the desired file and quit the session again. VIM allows you to do this i 1 step, using SCP. On your shell type vim scp://user@server//path/to/the/file and VIM will edit the file directly through SCP.

Recover the last visual selection

It often happens, that you selected some text in visual mode, e.g. to perform a regex on it. Occasionally your regex is incorrect and you have to revert the changes or they don't even happen. What you then need is to re-select the text portion again, which might take quite a long time, if you deal with large files. Hitting gv in command mode, takes that work from you and selects the last visual selection again.

Grep within VIM

Refactoring happens quite often in dynamic projects. "Real" IDE's offer a lot of tools for this task. While VIM does not, you can enhance your refactoring process a lot with using regex and this tiny tip: To grep through a lot of files for certain strings and then perform changes on each occurrence, vim offers the :grep command. :grep <string> <files> and VIM will store the list of changes for you and :cn will take you to the next occurance of your string. The :grep command utilizes your systems instance of grep, or a custom command stored in the grepprg configuration variable. There is also an internal implementation of grep available in VIM, which you can use with :vimgrep /<regex>/<flags> <files>. The benefit of the external implementation is, that you can easily customize the command to use, e.g. if you need to ignore certain directories while grepping recursively (for instance .svn dirs). You can also get a list of all occurrences of your grep command, using :ll, which is actually a feature of VIMs quickfix tool. Simply select the next entry you want to jump to and hit return in the location list.

Placeholders in commands

Who doesn't know the case, where you need to open a file in the same directory, the currently edited file resides in? Sure, you can simply use :split <file>, but if your current working directory is far away from the edited file, you possibly will have to type a long long path again. VIM offers you, to replace certain placeholders in commands automatically, most important: % is replaced with the actual file name (so, :split % will open a new window editing the current file - I know that just :split does the same, but it's a nice example to show what % does). So, how does this help in the described case? The replacement text can also be modified by certain modifiers, like :h, which removes the last portion of a file path (the file name). For instance if you currently edit /home/dotxp/dev/ez/ezc/trunk/ConsoleTools/src/table.php and want to split to /home/dotxp/dev/ez/ezc/trunk/ConsoleTools/src/output.php you can do this easily using :split %:h/output.php. Another useful modifier is :s?<regex>?<replacement>?, which obviously uses a regex to modify the given path. Unlikely the usual replacement command :s, you need to specify the global modifier g before the regex instead of doing so after it. A short example: :split %:gs?trunk?/releases/1.1/? will split to the file table.php of ConsoleTools release 1.1.

Emergency help

Appending an exclamation mark (!) to a command (:<command>!) forces VIM to execute the command, even if it might be dangerous to do so. Quite useless, but still funny is :help!.

Rewrap my text

I reached a stage, where do not only edit my source code using VIM, but mainly every other kind of text data (like documentation, todo lists, etc.), because I'm simply so used to its features. When editing plain text, the automatic wrap features (e.g. textwidth) of VIM are quite useful. In some cases, you might edit text later and therefore destroy a nicely wrapped text block. VIM can easily re-perform the wrapping for you: Select the text block in visual mode (using <SHIFT>+<v>) and hit "<g><q>".

Spelled correctly?

In VIM versions earlier to 7.0 you could add spellchecking using external scripts. The new version has this feature per default. What you need to do before spell checking is available, is to set the language of the document you are currently editing, using :setlocal spell spelllang=en_us (note, :setlocal only sets the value for the current window, use :set to set it globally). Now VIM highlights all unknown words and you have 3 important new commands available (in command mode). All are used when your cursor resides on a highlighted word: <z><=> opens a list of suggested replacements, you can choose one by typing its number or leave the selection untouched using <RETURN>. <z><g> adds the highlighted word to the global dictionary, so that it is recognized as correct on any spell checking session, later. <z><G> only adds the word to the local dictionary, so that is recognized as correct during the actual spell checking session.

Macros

Performing complex tasks very often might be quite time consuming and often not solvable using even highly complex regex. Before writing a complex script, you should consider, that VIM can simply follow your instructions and record them to a script: Type <q><a> in command mode, to instruct VIM to record all following commands to the register a. Perform whatever you like and finally hit <q> again (in command mode). Your macro is now recorded. Type <@><a> in command mode, to execute the contents of register a (namely your just recorded macro). A simple example for this is to move certain lines of a file to its top: Go to the first line you need. Type <q><a>. Now perform the changes: <d><d> to delete the line, <m><'> to set the "last editing position mark", <:><1><RETURN> to jump to the first line of the file, <P> to paste the just deleted line, <'><'> (2 single quotes) to return to the last editing position. Finally hit <q> to end the recording. Now you can just do <@><a> on every line you need at the start of your file. Easy, isn't it? :) Additionally <@><@> repeats the last macro execution (so you even don't need to remember the register you stored it in).

Uppercase letters

There are a lot of helpful shortcuts on your keyboard, which you might not know about. Every VIM user should now, that <d> deletes something, <y> (yank) copies something and <c> changes something (delete and start inserting) in VIM (all in command mode). Furthermore, most people know that e.g. <d><w> deletes the next word, etc.. What I did not know for a long time was, that <D>, <Y> and <C> are shortcuts to perform the desired operation until the end of the current line. For instance, simply type <C> to change the contents of the current line from your cursors position on.

Where am I?

If you have a large resolution (e.g. mine is 166x1200), you sometimes loose your cursor when looking at something else in the room. The new customization features of VIM 7 allow you to add a nice finding help: Add the lines autocmd InsertLeave * se nocul and autocmd InsertEnter * se cul to your .vimrc and you will get the current line underlined as soon as you enter insert mode.

Regex driven commands

Regular expressions belong to every VIM users toolbox. What some people might not know (so didn't I) is, that you can not only do replacements with them in VIM, but also address lines you want to perform commands on. Simply do :s/<regex>/=<command> to execute the given <command> on each text portion that matches <regex>.

Magic regex

Those of you who are used to PCRE syntax (and I hope all of you PHP developers out there), will often struggle over VIMs POSIX style regex and forget toescape parenthesis and other stuff. Usually VIMs regex work in so called magic mode, which means that VIM treats only a limited set of characters as magic inside a regex (that applies e.g. to ., matching any character, and * as a quantifier). Other characters, like (, ) and | are per default not considered magic and have to be escaped to make them magic (i\(, \) and \|). While this might be convenient insome cases (like simple text replacements without back-references), it becomes quite ugly, if you need to refer to a lot of capturing parenthesis. For this cases, it makes sense to switch VIM into very magic mode, using \v at the beginning of your regex. The opposite of this switch is \V, which makes the regex very nomagic. Because VIM is normally in magic mode (which corresponds to setting \m, if you change the default behaviour), there is also a nomagic mode, which considers less, but still some, characters as magic. Some simple examples:

This were quite many tips, especially for unexperienced VIM users. I recommend to not try to learn them all at once. Pick out 2-3 tips that you consider very important for your daily work and start getting used to them. When you are sure, you use them every time it makes sense, grab the next 2-3 and so on. I also recommend to every programmer out there, to read a work I personally call "Bible of text editing", written by Bram Moolenaar, the author of VIM itself. These 7 tips really make sense and explain quite simple, why I personally love to use VIM.

I hope the above shown tips help some people to make their daily work a bit more efficient and I also hope that I will manage to set up a new website soon, so I can consolidate all of my VIM tips in 1 central place. So long you will have to use my blog to find them again, so here is the whole collection of my "Comfortable PHP editing with VIM" series:

Part -1-
Part -2-
Part -3-
Part -4-

Note, that in the links in earlier editions of this series might be broken. Please refer to the latest versions of the links, contained in this article!

Have fun! :)

I recently uploaded a new version of PDV (phpDocumentor for VIM), which allows you to comfortable generate documentation for your PHP classses, functions, methods and attributes. The new version fixes some issues with the usage of tabs in your code (since I only use spaces, I did not notice these), as well as some smaller bugs. Enjoy! :)

It's time to continue my series, since I again updated my .vimrc in the past weeks for much more comfortable editing of PHP sources using VIM. There are several new features I added:

• Auto completion of functions using <TAB>
• Auto reloading of .vimrc when changed
• Enclose visually selected text by braces/quotes
• Much more convenient scrolling through autoclosing folds
• Search the PHP manual directly in VIM using phpm
• Added settings for misconfigured machines
• More convenient creation of phpdoc blocks with correct indenting and automatic * chars

To use some of this (and the older) features, you will have to download Rasmus' function list and install phpm (which is very easy).

Thanks a lot to everyone on the web providing me with support/tips!

For detailed information on what has changed, please read the extended entry.

^{A short hint for those of you who fight with auto completing enclosing chars (braces/qutotes) when you only want 1 exemplar: Use <CTRL>-v + the char to get only 1 of it!}

Sadly I lost a link I found while browsing the web where a script allows you to jump to PHP compile errors when running PHP CLI from VIM. Maybe someone can help me out?

Update: There were some <CR> missing in the <CTRL>-p mapping for running PHP CLI on the current file. I fixed that and replaced the download with version 1.1pl1. If anyone finds more bugs, please comment on this blog entry or email me!